[m-rev.] for review: parsing .used files

[Zoltan Somogyi]

2021-04-19 16:35 GMT+10:00 "Julien Fischer" <jfischer at opturion.com>:
>> Separate out the parsing of .used files.
>> > compiler/recompilation.check.m:
>>     The main predicate of this module, should_recompile_3, used to interleave
>>     the parsing the contents of the .used file, and checking which modules,
> 
> Delete the first "the" on that line.
> 
>>     if any, need to recompiled. And the parsing code itself was split up
> 
>    ... to be recompiled.

Both done.

>> +    % Alternatively, the .used file could contain two terms, the version
>> +    % number info, and everything else. We could then select the predicate
>> +    % we use to read in everything else based on the version number.
> 
> Using a separate initial term for the version number would be my preference.

OK. This happens to be relevant right now, because since I sent that diff,
I have started work on changing how recompilation.usage.m works.
It has traditionally also interleaved (a) figuring out what information
should go into the .used file, and (b) actually writing it out. My current
incomplete diff changes this so that it first constructs a single term
that represents the information to be written out, and then just writes it out.
This incomplete diff now bootchecks with one obvious exception:
since the code writing out .used files has switched to a new format
but the code reading them in hasn't, all the recompilation tests fail.

The type that defines my proposed new file format is attached.
I think it is a reasonable compromise between the our needs
as compiler writers for expressive function symbol names,
without making those names too long when we have to look at
.used files themselves. (Neither of which users have to do.)
I look forward to your feedback on this design.

A couple of other questions about next steps.

First, my diff cannot be committed without updating the code
that reads in and uses the contents of .used files. That second
part means updating the code in recompilation.check.m to work
based on the terms of the type in the attachment. Supporting
backward compatibility would require adding code to transform
the existing data structure to this one, which is a nontrivial amount
of work. Is there any point in doing this work? I don't think so,
since the recompilation package is set up so that any syntax error
in a .used file is handled by simply rebuilding everything,
and having the .used file contain a new format is a guaranteed
syntax error (since the version number will change, even if
nothing else does). But I am willing to be persuaded that backward
compatibility is needed.

The second question is about timestamps. We currently represent them
as strings that have to fit a strict format: yyyy-mm-dd hh:mm:ss.
I was thinking about a du representation, with six integer fields,
most of which can be uint8. I see no downside to this change;
does anyone else?

Another aspect of this is that a du type has room for an OS native 
representation of time. For unix, that would be seconds since 1970 jan 1;
I don't know what Windows and other OSs use.  For .used files, we could
write out both the OS native representation, which would mean nothing
to humans, and the yyyy-mm-dd hh:mm:ss that they can read, and
then pay attention to only the native representation after reading it
back in, unless that reading takes place on a different OS, in which case
we would compute the native representation from the human-readable
one just the way we do now.

And yet another aspect that we should think about is the inclusion
of sub-second-resolution time information. Some OSs now support
nanosecond resolution in e.g. file modification times, though of course
not all of that resolution is useful yet.

I look forward to your opinions.

> The diff itself looks fine.

Thank you.

Zoltan.
-------------- next part --------------
A non-text attachment was scrubbed...
Name: used_file
Type: application/octet-stream
Size: 2837 bytes
Desc: not available
URL: <http://lists.mercurylang.org/archives/reviews/attachments/20210419/2a895233/attachment.obj>