[m-dev.] [m-users.] Comparison in the reference manual

[Julien Fischer]

Hi Mark,

On Tue, 17 Nov 2015, Mark Brown wrote:

>>> float: We can at least say that < and > never contradict the
>>> mathematical ordering, even if = can due to lack of precision. Can we
>>> assume implementations may use +/- infinity?
>>
>>
>> If we are assuming IEEE 754 floats, yes.   For all of our current
>> backends that is definitely yes.
>
> For ints and floats I've decided to keep it conservative, and just say
> that implementations are allowed to be inconsistent if literals are
> out-of-range.

I think that's too conservative.  The implementation should just reject
int literals that will overflow (i.e. what is does now).  For systems
with IEEE floats we should replace overflows with +/- infinity.  What to
do on systems that don't provide IEEE fp is a more open question, but we
don't currently have any of those.

Question: should we just require the use of IEEE floats for Mercury?

> diff --git a/doc/reference_manual.texi b/doc/reference_manual.texi
> index f5c7e51..971f938 100644
> --- a/doc/reference_manual.texi
> +++ b/doc/reference_manual.texi

...

> +For the builtin type @code{float},
> +the standard ordering approximates the usual numerical ordering.

I see what you did there!

> +If the result of @code{builtin.compare/3} is @code{(<)} or @code{(>)}
> +then this relation holds in the numerical ordering,
> +but this is not necessarily the case for @code{(=)} due to lack of precision.

Ignoring the matter of NaNs for a moment, the order for floats should be
specified as it is in, for example, Java and C#:

   -infinity < -max < ... < -min < -0.0 == +0.0 < +min < ... < +max < +infinity


Not-a-number (NaN) values complicate matters.  The current implementation of
builtin.compare/3 is broken, for example the following evaluates to true:

     NaN = 0.0 * infinity,
     compare((=), NaN, 0.0)

but:

     NaN = 0.0 * infinity,
     unify(NaN, 0.0)

evaluates to false.

The IEEE 754-2008 standard (section 5.10) defines a total ordering on float
values, but I don't think that will work for us.  Java's Double.CompareTo
method treats NaN values as equal to themselves and greater than all other
float values (including +infinity), mainly so that you can use them in data
structures.  IIRC, C# does something similar.

My inclination is that we should simply make builtin.compare/3 throw an
exception for NaN values on the basis that they are not ordered w.r.t other
float values (or indeed themselves).

...

> +For the builtin type @code{char}, the standard ordering is
> +the numerical ordering of the Unicode code point values.

For completeness, you should say something about strings here too (e.g.
what's currently in the comment at the head of the string module.)

Julien.