[m-rev.] for review: merge integer token representations in the lexer

[Zoltan Somogyi]

On Sat, 22 Apr 2017 15:19:47 +1000 (AEST), Julien Fischer <jfischer at opturion.com> wrote:

> diff --git a/library/lexer.m b/library/lexer.m
> index 822f2de..c48a800 100644
> --- a/library/lexer.m
> +++ b/library/lexer.m
> @@ -30,11 +30,7 @@
>   :- type token
>       --->    name(string)
>       ;       variable(string)
> -    ;       integer(integer_base, int)
> -
> -    ;       big_integer(integer_base, integer)
> -            % An integer that is too big for `int'.
> -
> +    ;       integer(integer_base, integer, signedness, integer_size)

What is your reason for putting the value field *among* the
non-value fields? I would have thought that putting the value
either first or last would be conceptually cleaner.

> -    ;
> -        Token = big_integer(LexerBase, Integer),
> +        Signedness = lexer_signedness_to_term_signedness(LexerSignedness),
> +        Size = lexer_size_to_term_size(LexerSize),

Why is there a need for these type conversions?
By that I mean: why does lexer.m has its own copies
of these types?

I haven't checked the updated expected outputs, but the rest is fine.

Zoltan.