[m-rev.] for review: Add a `pregen' grade component.

[Julien Fischer]

Hi Peter,

This change is fine by me, at least I can't think of a cleaner solution
to the problem.  (I agree with Paul, we should try to avoid having multiple
source distributions.)

On Fri, Mar 15, 2013 at 5:48 PM, Peter Wang <novalazy at gmail.com> wrote:
> Still needs more testing.

I'm happy to help out with this if needed.

> Add a `pregen' grade component and associated option `--pregenerated-dist',
> intended for use in the pre-generated C files in the source distribution
> ONLY.
>
> Traditionally, by forcing the use of 2 tag bits and boxed floats, we
> could provide one set of pre-generated C files, nominally targeting
> 32-bit platforms but also working on 64-bit platforms.
> This became insufficient after I made two data representation changes.
> The same constructor may have different layouts of its arguments
> for --bits-per-word=32 and --bits-per-word=64:
>
>   - double-precision `float' arguments can be stored across two
>     words on 32-bit, whereas only one word is required on 64-bit;
>
>   - consecutive enum arguments can be packed into a single word
>     but the number of arguments that can be packed differs.
>
> As a result, data structures created in a user module could be
> misinterpreted when passed to a pre-generated standard library module,
> and vice versa.  (The enum case probably does not occur in practice.)
>
> The solution adopted here is to allow configure to detect normal 64-bit
> settings (3 tag bits, unboxed floats) irrespective of whether a usable
> Mercury installation already exists.  When present, the `pregen' grade
> component causes the compiler and scripts to override the configuration
> and enforce settings for portable C source files.  The source
> distribution should supply C source files from a `pregen' grade.

We should probably switch the source distribution around to use the high-level
C backend at this point as well, i.e. use hlc.gc.prege as the bootstrap grade.
(Support for pre-generated C files in the hlc grade has been present for
several years now.)

This should reduce the instances of users bumping into C compiler bugs and will
probably result in a faster compiler.  (hlc.gc is usually the fastest grade for
the Mercury compiler.)

> During installation, if required, the pre-generated C source files are
> used to build and install a Mercury compiler _in a .pregen grade_.
> Then it is used to install the libraries _in non-.pregen grades_,
> so that configured settings have their usual effect.

That of course presupposes that the pregen version of the compiler is
statically linked against the Mercury libraries (i.e.
--mercury-linkage=static).  It's probably not worth supporting a pregen version
of the Mercury compiler that is linked against the shared version of the Mercury
libraries.  (That's not the case for a non-pregen version of the
Mercury compiler
however; we still want --mercury-linkage=shared to be possible in that case.)

> Another benefit is that a user on a 64-bit system will get the "optimal"
> installation using 3 tag bits and unboxed floats without additional
> effort.
>
> A small disadvantage is that a minimal installation of Mercury takes
> slightly longer, as the default library grade no longer comes from the
> pre-generated C source files.

Something we Mac OS X users of Mercury are already used to ;-)

> compiler/options.m:
>         Add `--pregenerated-dist' option (same as `pregen' grade component).
>
>         Add `--arg-pack-bits <n>' internal option.
>         `--allow-argument-packing' is obsolete.
>
>         Add `--allow-double-word-fields' internal option.
>
> compiler/handle_options.m:
>         Add `pregen' grade component.  `pregen' implies boxed floats and
>         `spf' (single-prec float) implies unboxed floats, so we consider
>         them incompatible.  There should be no need to use both at once.

(We really should add float32 and float64 and get rid of the spf grades
entirely.)

>         In `pregen' grades, override settings to create portable C
>         source files.
>
>         Handle the internal `--arg-pack-bits <n>' option.
>
> compiler/make_hlds_passes.m:
>         Pack only as many consecutive arguments as will fit into the
>         number of bits specified by `--arg-pack-bits', rather than into
>         the number of word bits.  `pregen' implies --arg-pack-bits=32
>
>         Use `--allow-double-word-fields' to decide whether to store
>         floats across two words.  `pregen' disables it.
>
> compiler/compile_target_code.m:
>         Define MR_PREGENERATED_DIST when compiling C files in a `pregen'
>         grade.
>
> compiler/c_util.m:
>         Write PREGENERATED_DIST={yes,no} into the grade info header at
>         the top of C files.  This is checked by configure.
>
> runtime/mercury_conf_param.h:
>         Make MR_PREGENERATED_DIST force the same settings as the
>         `--pregenerated-dist' option.
>
> runtime/mercury_grade.h:
>         Add "_pregen" to the MR_GRADE macro so that `pregen' and
>         non-`pregen' object files are not accidentally mixed.
>
> configure.ac:
>         As above, do NOT force 2 tag bits and unboxed floats when there
>         is no usable Mercury compiler installed.
>
>         BOOTSTRAP_GRADE is now a grade containing `.pregen'.
>
>         Make a minimal installation install $BEST_GRADE_FOR_COMPILER as
>         the default grade, not the grade that the compiler happens to be
>         built in, which might be a `pregen' grade.
>
> scripts/canonical_grade.sh-subr:
> scripts/init_grade_options.sh-subr:
> scripts/mgnuc.in:
> scripts/parse_grade_options.sh-subr:
>         Support `pregen' grade component and `--pregenerated-dist'
>         in shell scripts.
>
> tools/test_mercury:
>         Add a note.

...

> diff --git a/runtime/mercury_conf_param.h b/runtime/mercury_conf_param.h
> index 1e17739..170d9f2 100644
> --- a/runtime/mercury_conf_param.h
> +++ b/runtime/mercury_conf_param.h
> @@ -133,6 +133,7 @@
>  ** MR_USE_MINIMAL_MODEL_STACK_COPY
>  ** MR_USE_MINIMAL_MODEL_OWN_STACKS
>  ** MR_MINIMAL_MODEL_DEBUG
> +** MR_PREGENERATED_DIST
>  ** MR_USE_SINGLE_PREC_FLOAT
>  ** MR_EXTEND_STACKS_WHEN_NEEDED
>  ** MR_STACK_SEGMENTS
> @@ -159,6 +160,7 @@
>  **             --reserve-tag
>  **             --use-minimal-model
>  **             --minimal-model-debug
> +**             --pregenerated-dist
>  **             --single-prec-float
>  **             --extend-stacks-when-needed
>  **             --stack-segments
> @@ -667,6 +669,22 @@
>  */
>
>  /*
> +** MR_PREGENERATED_DIST overrides configured values to assume values compatible

s/assume/take

> +** with those used to generate the pre-generated source distribution.

  ... pre-generated C files in the source distribution.

> +** It implies boxed floats so is incompatible with single-precision floats,
> +** which imply unboxed floats.
> +*/

...

> diff --git a/runtime/mercury_grade.h b/runtime/mercury_grade.h
> index 7c12d48..44d356b 100644
> --- a/runtime/mercury_grade.h
> +++ b/runtime/mercury_grade.h


> diff --git a/tools/test_mercury b/tools/test_mercury
> index 23a3e73..c63f171 100755
> --- a/tools/test_mercury
> +++ b/tools/test_mercury
> @@ -881,6 +881,7 @@ case $HOST in $ROTD_HOST)
>      rm -f so_locations &&
>      rm -f .enable_lib_grades &&
>      $ACLOCAL_CMD &&
> +    # The cache variable settings can be replaced by --pregenerated-dist.
>      autoconf &&
>      mercury_cv_low_tag_bits=2 \
>      mercury_cv_bits_per_word=32 \

I intend to separate out the stuff for building the source dist. into a new
shell script, but that's fine for now.  (Actually, I've been building the
source distributions since the move over to git using my own script; once this
change has bootstrapped, I'll switch my version over to use --pregenerated-dist
and post it for review.)

Cheers,
Julien.