[m-dev.] issues with installing libraries

[Julien Fischer]

Hi Zoltan,

Apologies for the belated reply.

On Fri, 13 Sept 2024 at 19:09, Zoltan Somogyi <zoltan.somogyi at runbox.com> wrote:

> > > Issue 6:
> > >
> > >     The directory paths we install grade-specific files in
> > >     include one directory name component that represents the grade,
> > >     and another that represents the target architecture. The latter
> > >     is clearly useful for execution and library files that contain
> > >     machine code, in that they detect and report attempts to use
> > >     e.g. an x86-64 library on an Arm machine. However, not *all*
> > >     grade-specific files contain machine code. For these, the directory
> > >     name component that identifies the target architecture is unnecessary,
> > >     though it is not harmful. It can even be very slightly helpful in that
> > >
> > >     - installing a library in e.g. /path/to/install_dir on one architecture,
> > >     - and then installing a slightly different version of that same library,
> > >       also to /path/to/install_dir, for a different architecture,
> > >
> > >     will not overwrite the existing, old versions of those grade-specific
> > >     files, and each grade-and-architecture-specific directory will contain
> > >     grade-specific non-machine-code files and grade-specific machine-code
> > >     files that are from the same version and should thus be consistent
> > >     with each other.
> > >
> > >     However, this helpfulness is useless, because the second install
> > >     with mmc -mmake *will* overwrite the old version's installed
> > >     *non*-grade-specific files, leaving them out of sync with the first
> > >     install's grade-specific files. When installing with mmake, nothing
> > >     should be out of sync, because mmake's install rules will *delete*
> > >     the target directories before recreating and populating them.
> > >
> > >     QUESTION Does anyone remember any occasion where we said or promised
> > >     anything to users about the outcome of any attempt to install to an
> > >     already-existing directory?
> >
> > No.
>
> Thanks. If Julien does not remember any such promise either, then
> I can rationalise the install directory structure.

I would expect such a promise to be mentioned in the user's guide.
It isn't, so we didn't ;-). (I don't recall such a promise being made
elsewhere either.)
(Actually, I think we should explicitly warn against such a practice
as you can potentially
end up with files related to deleted modules etc. remaining present in
the installation
directory if you're not careful.)

> Julien, do you plan to reply at some point?

I don't have anything particularly illuminating to add.  I spent a bit
of time last weekend
trying to understand how we arrived at the current library
installation structure only to
arrive at the conclusion that most of it was a fait accompli before it
even arrived in version
control.  (Support for user libraries with mmake was added ~1999, the
structure they
use mimics that used for the Mercury libraries themselves. The
structure used for the
Mercury libraries was mostly in place around 1995.)

In terms of library installation structure, I think there are few
principles we have been
trying to / should adhere to:

1. A library installation should be usable using (a) mmake, (b) mmc --make and
   (c) mmc invoked on a single module regardless of which build system is used
   to build it.  (Obviously, libraries installed by mmc --make cannot
have .trans_opt
   files, but the compiler can already handle the absence of those.)

2. Users should not have to understand the structure of a library installation.
    If a user is using a Mercury library via the Mercury compiler,
then the compiler
    understands this structure and all the user should need to do is
point it to the
    correct library and installation directory (e.g. using the --ml
and --mld options
    or the equivalent mmake variables.)
    If the user is using a Mercury library not via the Mercury
compiler (e.g. a C++
    program that links against Mercury libraries or refers to Mercury
headers), then
    the correct way is to get the Mercury compiler to tell the user
what information
    they require.  We already have a bunch of options,
--output-cflags, --output-c-include-dir-flags,
   --output-grade-defines, that do exactly this.

We should add something to compiler/notes documenting whatever the
library installation
structure is / becomes.  (Although I am fairly certain you are
intending to do that anyway.)

Julien.