[m-rev.] comments: non-blocking lex

[Ralph Becket]

Michael Day, Monday, 15 October 2001:
> 
> While lex provides "the holy grail of piecemeal lexing of stdin", this
> patch allows it to do so without blocking when there is no input to read.
> It should not break backwards compatibility in most cases, but allows read
> predicates to return blocked if no input is available and provides a
> read_no_block that will return the next token if enough input is present
> to create one.
> 
> This is convenient if you wish to tokenise multiple input sources
> simultaneously without blocking (sockets for example). There is the
> potential for an infinite loop however if read is called and the read
> predicate persists in calling blocked.

Isn't this what spawn is for?

I'd be reluctant to complicate the interface to lex when a reasonable
alternative solution already exists.

> Additionally, this patch adds a variant of manipulate source that lets you
> thread another accumulator through, for example when the source is *not*
> the io state but you wish to manipulate it *using* the io state.

I've never been that happy with lex__manipulate_source/3.  It is useful
when lexing IO streams, but for strings etc. it's not entirely clear to
me what the manipulating predicate should see.  Right now, for strings,
you see the entire string as provided to lex__start/2.  Perhaps it would
make more sense to just see the remainder of the string that has not been
parsed by the lexer.  However, the solution should also work for lists
etc.  Perhaps a typeclass based solution would be better.

- Ralph
--------------------------------------------------------------------------
mercury-reviews mailing list
post:  mercury-reviews at cs.mu.oz.au
administrative address: owner-mercury-reviews at cs.mu.oz.au
unsubscribe: Address: mercury-reviews-request at cs.mu.oz.au Message: unsubscribe
subscribe:   Address: mercury-reviews-request at cs.mu.oz.au Message: subscribe
--------------------------------------------------------------------------