[m-dev.] making .trans_opt files in parallel

[Zoltan Somogyi]

2022-12-12 16:04 GMT+11:00 "Peter Wang" <novalazy at gmail.com>:
> I would like to fix the problem of .trans_opt files for the standard
> library largely being made sequentially, instead of in parallel.
> 
> As a reminder, the standard library has a large cycle of modules that
> import each other. The compiler imposes an arbitrary ordering on the
> modules in the cycle so that "earlier" modules will not try to read from
> the .trans_opt files of "later" modules before they have been made.
> Thus the .trans_opt files can be made one by one, but not in parallel.
> 
> My idea is to introduce a mechanism to manually drop edges from the
> .trans_opt dependency graph. For example, a pragma in module M of the
> form:
> 
>     :- pragma no_trans_opt(foo).
> 
> would tell the compiler that it cannot read from foo.trans_opt
> when making the M.trans_opt file.

I don't think this is the best way to achieve your objective.

What we want to specify is effectively a dag; if module B is reachable
from module A, then module A may depend on B.trans_opt; otherwise,
it may not. Each no_trans_opt pragma would put only one constraint
on the dag, and there would be no easy way to check that two such
pragmas in different modules basically rule out both A using B.trans_opt
and B using A.trans_opt, which would not be a correctness problem,
but may be a performance problem.

I think it would be better to specify the whole dag all at once,
probably as the contents of a file, whose name can be passed
to mmc in an option. Unfortunately, I don't know of a simple way
to encode dag in a text file in a way that is

- unambiguous,
- easy to read and understand,
- but does not reduce all dags to a total order.

> A module's .d file contains a rule for the .trans_opt_date file
> that both tells mmake when the .trans_opt needs to be regenerated,
> *and* is read back by the Mercury compiler to tell it which
> .trans_opt files it is allowed to read when making a .trans_opt file.
> 
> The pragmas would remove entries from the .trans_opt_date rule
> (and perhaps that is enough?)
> 
> Zoltan, in reply to my previous message on this topic on 2019-08-05,
> you wrote:
> 
>> What I have long wanted was the ability to *visualise* the cyclic (and other)
>> dependencies graphically, with the info about *what* module A needs from
>> module B being available if you click on the arrow between them.
>> The compiler can now be asked to generate the info for the links;
>> the visualization part is still missing.
> 
> Do you remember what information you were referring to?

The compiler should output (parts of) the what-imports-what graph
when given the --imports-graph option. By gather up the pieces
of this graph from all modules imported by e.g. library.m, we *should*
be able to generate the dag I mention above. Users could then tinker
with the file containing the dag, if they wish,

What the imports graph is missing is *why* module A imports module B.
The --show-dependency-graph option can be viewed as a start on this.
 It does not tell you *everything* that module A needs from module B,
but it does an important part of that job. To make this useful, one would
need to finish that job, and then find a way to present the info to users
in a way that is easy to use. This is not easy, because users will probably
be interested in only a small part of the data, and would probably be
overwhelmed if given *all* of it.

Zoltan.