[m-users.] Lex & Yacc for Mercury

[Peter Wang]

On Sat, 31 Dec 2016 14:25:49 +1100, Paul Bone <paul at bone.id.au> wrote:
> 
> We have a lex/flex like library called lex.
> 
> https://github.com/Mercury-Language/mercury/tree/master/extras/lex
> 
> I'm currently using this within the Plasma compiler.  I found that it would
> use a lot of memory, when I restrected it to ASCII or ISO-8859 it fixed the
> problem.  The copy I'm using in Plasma is here:
> https://github.com/PaulBone/plasma/tree/master/src  I can't find the code I
> needed to change.  Let me know if you need it and I'll have a better look.

Hi,

It's been a while since I looked at the lex module.  I think the memory
usage was due to use of sparse_bitset to represent charsets, where a
better choice would be diet.  The lex module also creates the entire
DFA upfront, while it is possible for a regular expression engine to
materialise DFA states on demand.  (I was working on another regexp
implementation for fun; perhaps I will brush it off some time.)

I would suggest to use a mature regular expression implementation
(written in C or whatever) that has had a lot more effort put into it.

Peter