[m-rev.] for review: allow optional underscores in numeric literals

[Julien Fischer]

On Wed, 11 Jan 2017, Paul Bone wrote:

> On Wed, Jan 11, 2017 at 02:07:55PM +1100, Julien Fischer wrote:
>>
>>>> diff --git a/library/lexer.m b/library/lexer.m
>>>> index c527ffd..fddd8aa 100644
>>>> --- a/library/lexer.m
>>>> +++ b/library/lexer.m
>>>> @@ -1912,13 +1976,19 @@ get_zero(Stream, Token, !IO) :-
>>>>         )
>>>>     ).
>>>>
>>>> +:- type last_digit_is_underscore
>>>> +    --->    last_digit_is_underscore
>>>> +    ;       last_digit_is_not_underscore.
>>>> +
>>>
>>> Could you add a comment explaining the purpose of this type?  I
>>> mistakeningly thought that the above code was saying what the last digit of
>>> the entire token was, rather than saying what the last digit _that it had
>>> seen_ was.  Thanks.
>>
>> Ok, how's this:
>>
>>      This type records whether the last digit seen by the lexer as it
>>      processes a numeric token was an underscore or not.
>>      Note that there may be other intervening characters in the token
>>      between the last digit and the current one (e.g. the decimal point
>>      in a float literal.)
>
> The only thing I'd add is "...so that invalid literals can be rejected" or
> similar.  No need to re-send this for review.

Rejecting invalid literals / tokens is the point of everything in that
module!  I've said that it's so we can detect invalid uses of
underscores in numeric literals.

Julien.