[m-rev.] Changes to extras/lex checked in.

[Ralph Becket]

I've checked in Holger's changes to lex.  Lex looks like a rather
handy tool now.

I had some problem checking in extras/lex/samples/demo.m, which
I've attached.  If someone could see if that made it in for me
I'd be grateful.

- Ralph



Estimated hours taken: 20
Branches: main

These changes were made by Holger Krug <hkrug at rationalizer.com> and
checked in by Ralph Becket.

The interface to lex has received a complete overhaul: unfortunately
any old code (was there any?) using lex will have to be changed to
suit.

The syntax for defining regular expression has been simplified
through judicious use of type classes.

The annotated_lexeme type has been changed: rather than distinguishing
between value, noval and ignore tags, each lexeme is associated with a
function to turn the matched string into a token.

An argument to lex__init/3 allows the user to supply a predicate
indicating which tokens should be ignored in the input stream.

Various utility predicates and functions have been provided.

lex.buf.m:
lex.convert.m:
	Minor cosmetic changes.

lex.lexeme.m:
	Cosmetic changes and alterations to deal with the new
	token reporting interface.

lex.regexp.m:
	Cosmetic changes and simplifcation of the regexp definition
	language.

lex.m:
	Changes to the API to deal with the new token reporting
	interface.  Some cosmetic changes.

COPYING.DOC:
COPYING.LGPL:
	Added to comply with the licence for these changes.

README:
	Updated to describe the new API.

samples/demo.m:
	Updated to use the new API.
-------------- next part --------------
A non-text attachment was scrubbed...
Name: demo.m
Type: application/octet-stream
Size: 3931 bytes
Desc: demo.m
URL: <http://lists.mercurylang.org/archives/reviews/attachments/20010815/46613956/attachment.obj>