[m-rev.] for review: merge integer token representations in the lexer

[Julien Fischer]

Hi Zoltan,

On Sat, 22 Apr 2017, Zoltan Somogyi wrote:

>
>
> On Sat, 22 Apr 2017 15:19:47 +1000 (AEST), Julien Fischer <jfischer at opturion.com> wrote:
>
>> diff --git a/library/lexer.m b/library/lexer.m
>> index 822f2de..c48a800 100644
>> --- a/library/lexer.m
>> +++ b/library/lexer.m
>> @@ -30,11 +30,7 @@
>>   :- type token
>>       --->    name(string)
>>       ;       variable(string)
>> -    ;       integer(integer_base, int)
>> -
>> -    ;       big_integer(integer_base, integer)
>> -            % An integer that is too big for `int'.
>> -
>> +    ;       integer(integer_base, integer, signedness, integer_size)
>
> What is your reason for putting the value field *among* the
> non-value fields? I would have thought that putting the value
> either first or last would be conceptually cleaner.

That order matches the order in which those things occur in integer
literals.

>> -    ;
>> -        Token = big_integer(LexerBase, Integer),
>> +        Signedness = lexer_signedness_to_term_signedness(LexerSignedness),
>> +        Size = lexer_size_to_term_size(LexerSize),
>
> Why is there a need for these type conversions?
> By that I mean: why does lexer.m has its own copies
> of these types?

The existing code already handled the base argument thus; the rationale
for it doing so was to avoid the lexer module having to import the term
module.

Julien.