[mercury-users] extras/lex: tokenizing string instead of stdin

[Ondrej Bojar]

Hi.

I'm trying to convince extras/lex/samples/demo.m to tokenize string
instead of stdin:

main -->
    io__print("Try me...
"),
    io__read_file_as_string(MaybeString),
    (
    { MaybeString = ok(String) },
      {
      Lexer  = lex__init(lexemes, lex__read_from_string, ignore(space)),
      State0 = lex__start(Lexer, String),
      tokenise_string(State0, State),
      RestStr     = lex__stop(State)
      }
    ;
    { MaybeString = error(_, Err) },
      io__write_string("I/O error: "), print(Err), nl
    ).

The lex__read_from_string is :- mode read_from_string(in, out, in, out) is
det, which is a pity, because read_pred is:

:- type read_pred(T)
    ==      pred(offset, read_result, T, T).
:- inst read_pred
    ==      ( pred(in, out, di, uo) is det ).

I still do not understand unique modes very well, except for passing
io__state, so I can't answer, whether I should wish to change
read_from_string/4 to be (in, out, di, uo) or whether I should wish to
change something different.

(Further on, I wish to use lex+moose, isn't there a sample that would
bind the two ready?)

(I've modified lex.m slightly to return 1. the result and 2. the offset of
the current token, even in case of success, as a success at the lexer can
still happen to be an error at the parser, and it's important to report
the error in a human-readable form. The lex/README says a feature to add:
include line number as part of the offset. Isn't this ready yet? It's not
in my rotd-2002-10-07.)

What a pity I'm now used to mmc's way of reporting errors. mmc is quite a
high (and hard) standard to follow... I'm afraid moose is really too
immature to get close enough.

Thanks, Andrew.

--------------------------------------------------------------------------
mercury-users mailing list
post:  mercury-users at cs.mu.oz.au
administrative address: owner-mercury-users at cs.mu.oz.au
unsubscribe: Address: mercury-users-request at cs.mu.oz.au Message: unsubscribe
subscribe:   Address: mercury-users-request at cs.mu.oz.au Message: subscribe
--------------------------------------------------------------------------