[m-rev.] for review: Add sanity check to tools/import_srcdist. (Peter Wang)

[Fabrice Nicol]

The best "sanity check" would be to test-build the tagged packages, at 
least the upcoming ones.

This can be automated using GH Actions. If you find the idea useful, I 
could contribute some .yml code to the builder.

Fabrice


> Send reviews mailing list submissions to
> 	reviews at lists.mercurylang.org
>
> To subscribe or unsubscribe via the World Wide Web, visit
> 	https://lists.mercurylang.org/listinfo/reviews
> or, via email, send a message with subject or body 'help' to
> 	reviews-request at lists.mercurylang.org
>
> You can reach the person managing the list at
> 	reviews-owner at lists.mercurylang.org
>
> When replying, please edit your Subject line so it is more specific
> than "Re: Contents of reviews digest..."
>
>
> Today's Topics:
>
>     1. Re: diff: fix a bug in tools/import_srcdist (Peter Wang)
>     2. Re: diff: fix a bug in tools/import_srcdist (Julien Fischer)
>     3. Re: for review: remove anachronisms from parse_module.m
>        (Julien Fischer)
>     4. Re: diff: fix a bug in tools/import_srcdist (Peter Wang)
>     5. for review: Add sanity check to tools/import_srcdist. (Peter Wang)
>     6. Re: for review: Add sanity check to tools/import_srcdist.
>        (Julien Fischer)
>
>
> ----------------------------------------------------------------------
>
> Message: 1
> Date: Thu, 27 Jan 2022 13:50:37 +1100
> From: Peter Wang <novalazy at gmail.com>
> To: Julien Fischer <jfischer at opturion.com>
> Cc: reviews at lists.mercurylang.org
> Subject: Re: [m-rev.] diff: fix a bug in tools/import_srcdist
> Message-ID: <20220127135037.GD1371 at kola.localdomain>
> Content-Type: text/plain; charset=utf-8
>
> On Thu, 27 Jan 2022 01:33:08 +1100 Julien Fischer <jfischer at opturion.com> wrote:
>> Hi Peter,
>>
>> On Thu, 27 Jan 2022, Peter Wang wrote:
>>
>>> On Wed, 26 Jan 2022 23:52:09 +1100 Julien Fischer <jfischer at opturion.com> wrote:
>>>> Fix a bug in tools/import_srcdist.
>>>>
>>>> tools/import_srcdist:
>>>>       Add files with the -f (--force) option; otherwise files that
>>>>       match patterns in .gitignore will be ignored.
>>>>
>>> Ouch.
>> Indeed. I'm assuming git add changed its default behaviour at some point
>> since up until around September 2020 things were fine.  Ideally, the
>> script should also check that, say, compiler/mercury_compile.c has been
>> imported into git.
> I'll send a change for that later.
>
> Note: the results from my previous email had a few false positives
> because compiler/mercury_compile.c didn't exist in very old versions.
>
>>> I wrote a quick script to check which tags are affected.
>>> The results are below. I still have most, maybe all,
>>> of the "old" tarballs so I can re-import them.
>> I probably have some others.
>>
>>> I also have a smattering of the newer tarballs that I downloaded.
>>> Perhaps you have a more complete set somewhere?
>> I have some of them -- they're just rotd package, so having them
>> all is not that important.
> Right.
>
>>> Otherwise, we can just import what we have, and delete any broken
>>> tags.
>> I assume you can modify the script you have below to delete the
>> broken tags?
> I've reimported 141 archives into my workspace.
> Do you want me to force-push the fixed tags now?
>
> That leaves the following 176 tags witih missing files.
> Do you want me to delete those tags now?
>
> Peter
>
>
> rotd-2013-01-30 MISSING
> rotd-2015-02-02 MISSING
> rotd-2015-02-06 MISSING
> rotd-2015-02-12 MISSING
> rotd-2015-02-13 MISSING
> rotd-2015-02-18 MISSING
> rotd-2015-02-19 MISSING
> rotd-2015-02-27 MISSING
> rotd-2015-03-02 MISSING
> rotd-2020-10-11 MISSING
> rotd-2020-10-12 MISSING
> rotd-2020-10-15 MISSING
> rotd-2020-10-16 MISSING
> rotd-2020-10-17 MISSING
> rotd-2020-10-18 MISSING
> rotd-2020-10-21 MISSING
> rotd-2020-10-22 MISSING
> rotd-2020-10-23 MISSING
> rotd-2020-10-25 MISSING
> rotd-2020-10-26 MISSING
> rotd-2020-10-30 MISSING
> rotd-2020-10-31 MISSING
> rotd-2020-11-05 MISSING
> rotd-2020-11-06 MISSING
> rotd-2020-11-07 MISSING
> rotd-2020-11-10 MISSING
> rotd-2020-11-11 MISSING
> rotd-2020-11-14 MISSING
> rotd-2020-11-15 MISSING
> rotd-2020-11-16 MISSING
> rotd-2020-11-17 MISSING
> rotd-2020-11-20 MISSING
> rotd-2020-11-21 MISSING
> rotd-2020-11-23 MISSING
> rotd-2020-11-30 MISSING
> rotd-2020-12-06 MISSING
> rotd-2020-12-07 MISSING
> rotd-2020-12-15 MISSING
> rotd-2020-12-23 MISSING
> rotd-2020-12-29 MISSING
> rotd-2020-12-30 MISSING
> rotd-2021-01-01 MISSING
> rotd-2021-01-05 MISSING
> rotd-2021-01-07 MISSING
> rotd-2021-01-08 MISSING
> rotd-2021-01-10 MISSING
> rotd-2021-01-13 MISSING
> rotd-2021-01-22 MISSING
> rotd-2021-01-23 MISSING
> rotd-2021-01-25 MISSING
> rotd-2021-01-27 MISSING
> rotd-2021-01-29 MISSING
> rotd-2021-01-31 MISSING
> rotd-2021-02-01 MISSING
> rotd-2021-02-02 MISSING
> rotd-2021-02-14 MISSING
> rotd-2021-02-15 MISSING
> rotd-2021-02-20 MISSING
> rotd-2021-02-21 MISSING
> rotd-2021-02-24 MISSING
> rotd-2021-02-26 MISSING
> rotd-2021-02-27 MISSING
> rotd-2021-03-01 MISSING
> rotd-2021-03-05 MISSING
> rotd-2021-03-06 MISSING
> rotd-2021-03-07 MISSING
> rotd-2021-03-08 MISSING
> rotd-2021-03-22 MISSING
> rotd-2021-03-23 MISSING
> rotd-2021-03-24 MISSING
> rotd-2021-03-25 MISSING
> rotd-2021-03-28 MISSING
> rotd-2021-03-29 MISSING
> rotd-2021-04-01 MISSING
> rotd-2021-04-03 MISSING
> rotd-2021-04-04 MISSING
> rotd-2021-04-05 MISSING
> rotd-2021-04-09 MISSING
> rotd-2021-04-12 MISSING
> rotd-2021-04-13 MISSING
> rotd-2021-04-19 MISSING
> rotd-2021-04-20 MISSING
> rotd-2021-04-21 MISSING
> rotd-2021-04-22 MISSING
> rotd-2021-04-26 MISSING
> rotd-2021-04-27 MISSING
> rotd-2021-04-28 MISSING
> rotd-2021-05-01 MISSING
> rotd-2021-05-03 MISSING
> rotd-2021-05-06 MISSING
> rotd-2021-05-11 MISSING
> rotd-2021-05-17 MISSING
> rotd-2021-05-19 MISSING
> rotd-2021-05-21 MISSING
> rotd-2021-05-23 MISSING
> rotd-2021-05-25 MISSING
> rotd-2021-05-26 MISSING
> rotd-2021-05-28 MISSING
> rotd-2021-05-29 MISSING
> rotd-2021-06-01 MISSING
> rotd-2021-06-02 MISSING
> rotd-2021-06-03 MISSING
> rotd-2021-06-04 MISSING
> rotd-2021-06-05 MISSING
> rotd-2021-06-06 MISSING
> rotd-2021-06-07 MISSING
> rotd-2021-06-09 MISSING
> rotd-2021-06-13 MISSING
> rotd-2021-06-14 MISSING
> rotd-2021-06-15 MISSING
> rotd-2021-06-16 MISSING
> rotd-2021-06-17 MISSING
> rotd-2021-06-19 MISSING
> rotd-2021-06-20 MISSING
> rotd-2021-06-23 MISSING
> rotd-2021-06-26 MISSING
> rotd-2021-06-27 MISSING
> rotd-2021-06-28 MISSING
> rotd-2021-07-01 MISSING
> rotd-2021-07-04 MISSING
> rotd-2021-07-06 MISSING
> rotd-2021-07-07 MISSING
> rotd-2021-07-10 MISSING
> rotd-2021-07-11 MISSING
> rotd-2021-07-12 MISSING
> rotd-2021-07-20 MISSING
> rotd-2021-07-21 MISSING
> rotd-2021-07-22 MISSING
> rotd-2021-07-25 MISSING
> rotd-2021-07-26 MISSING
> rotd-2021-07-27 MISSING
> rotd-2021-07-28 MISSING
> rotd-2021-07-29 MISSING
> rotd-2021-07-30 MISSING
> rotd-2021-07-31 MISSING
> rotd-2021-08-01 MISSING
> rotd-2021-08-02 MISSING
> rotd-2021-08-03 MISSING
> rotd-2021-08-04 MISSING
> rotd-2021-08-06 MISSING
> rotd-2021-08-07 MISSING
> rotd-2021-08-08 MISSING
> rotd-2021-08-09 MISSING
> rotd-2021-08-10 MISSING
> rotd-2021-08-11 MISSING
> rotd-2021-08-12 MISSING
> rotd-2021-08-13 MISSING
> rotd-2021-08-17 MISSING
> rotd-2021-08-18 MISSING
> rotd-2021-08-19 MISSING
> rotd-2021-08-20 MISSING
> rotd-2021-08-21 MISSING
> rotd-2021-08-23 MISSING
> rotd-2021-08-24 MISSING
> rotd-2021-08-25 MISSING
> rotd-2021-08-28 MISSING
> rotd-2021-08-29 MISSING
> rotd-2021-08-30 MISSING
> rotd-2021-09-02 MISSING
> rotd-2021-09-07 MISSING
> rotd-2021-09-08 MISSING
> rotd-2021-09-11 MISSING
> rotd-2021-09-13 MISSING
> rotd-2021-09-17 MISSING
> rotd-2021-09-25 MISSING
> rotd-2021-09-28 MISSING
> rotd-2021-09-29 MISSING
> rotd-2021-09-30 MISSING
> rotd-2021-10-01 MISSING
> rotd-2021-10-02 MISSING
> rotd-2021-10-03 MISSING
> rotd-2021-10-11 MISSING
> rotd-2021-10-12 MISSING
> rotd-2021-10-15 MISSING
> rotd-2021-10-30 MISSING
> rotd-2021-10-31 MISSING
>
>
> ------------------------------
>
> Message: 2
> Date: Thu, 27 Jan 2022 14:49:20 +1100 (AEDT)
> From: Julien Fischer <jfischer at opturion.com>
> To: Peter Wang <novalazy at gmail.com>
> Cc: reviews at lists.mercurylang.org
> Subject: Re: [m-rev.] diff: fix a bug in tools/import_srcdist
> Message-ID: <dcce286c-a8ca-ec29-12cf-e2142bfbf73c at opturion.com>
> Content-Type: text/plain; charset=US-ASCII; format=flowed
>
>
> Hi Peter,
>
> On Thu, 27 Jan 2022, Peter Wang wrote:
>
>>> I assume you can modify the script you have below to delete the
>>> broken tags?
>> I've reimported 141 archives into my workspace.
>> Do you want me to force-push the fixed tags now?
> Yes.
>
>> That leaves the following 176 tags witih missing files.
>> Do you want me to delete those tags now?
> Yes, go ahead.
>
> Thanks for that.
>
> Julien.
>
>
> ------------------------------
>
> Message: 3
> Date: Thu, 27 Jan 2022 15:07:47 +1100 (AEDT)
> From: Julien Fischer <jfischer at opturion.com>
> To: Zoltan Somogyi <zoltan.somogyi at runbox.com>
> Cc: Reviews <reviews at lists.mercurylang.org>
> Subject: Re: [m-rev.] for review: remove anachronisms from
> 	parse_module.m
> Message-ID: <3dfddb65-6a83-3056-ad42-268f56518a4d at opturion.com>
> Content-Type: text/plain; charset=US-ASCII; format=flowed
>
>
> On Thu, 27 Jan 2022, Zoltan Somogyi wrote:
>
>> Remove anachronisms from parse_module.m.
>>
>> Delete a globals structure from many predicates' argument lists.
>> These were needed when we switched from printing out each error immediately
>> using io.write_string to constructing an error_spec and printing it out
>> immediately, but have not been needed since the code to print
>> the error_specs constructed here has been moved outside this module.
>> The only reason why the compiler could not tell us that these arguments
>> were not needed anymore is that they *were* being passed in recursive calls
>> (both self-recursive and mutually recursive).
>>
>> Delete two output arguments of check_for_unexpected_item_at_end that all
>> of its caller ignore.
>>
>> Delete the version_number_info_type. It has not been needed since we moved
>> version numbers items to *before* the ':- interface' marker in interface files.
>>
>> If trying to read in a version numbers item results in garbage, record
>> the error but do not try to read the *next* term in the interface file
>> as a version numbers item. If an automatically generated file such as
>> an interface file contains an error, the entirety of that file is suspect.
>> Any attempt at recovery from such errors is more likely to cause avalanche
>> errors than a clean recovery.
>>
>> Give two of the function symbols in the read_io_result type clearer names.
>>
>> Convert more predicates to returning *either* the entity it is looking for
>> *or* a description of the errors that prevent us from finding it,
>> but not both.
>>
>> Move the code that reads in one or both sections of an interface file
>> into a predicate of its own, in order to reduce indentation levels.
> That's fine.
>
> Julien.
>
>
> ------------------------------
>
> Message: 4
> Date: Thu, 27 Jan 2022 16:33:27 +1100
> From: Peter Wang <novalazy at gmail.com>
> To: Julien Fischer <jfischer at opturion.com>
> Cc: reviews at lists.mercurylang.org
> Subject: Re: [m-rev.] diff: fix a bug in tools/import_srcdist
> Message-ID: <20220127163327.GF1812 at lucy.localdomain>
> Content-Type: text/plain; charset=utf-8
>
> On Thu, 27 Jan 2022 14:49:20 +1100 Julien Fischer <jfischer at opturion.com> wrote:
>> Hi Peter,
>>
>> On Thu, 27 Jan 2022, Peter Wang wrote:
>>
>>>> I assume you can modify the script you have below to delete the
>>>> broken tags?
>>> I've reimported 141 archives into my workspace.
>>> Do you want me to force-push the fixed tags now?
>> Yes.
>>
>>> That leaves the following 176 tags witih missing files.
>>> Do you want me to delete those tags now?
>> Yes, go ahead.
>>
>> Thanks for that.
> Ok, done. Looks like you will want to run something like:
>
>      git fetch origin --prune --prune-tags
>
> to remove the stale tags from your workspace.
>
> Peter
>
>
> ------------------------------
>
> Message: 5
> Date: Thu, 27 Jan 2022 17:19:10 +1100
> From: Peter Wang <novalazy at gmail.com>
> To: reviews at lists.mercurylang.org
> Subject: [m-rev.] for review: Add sanity check to
> 	tools/import_srcdist.
> Message-ID: <20220127061910.26275-1-novalazy at gmail.com>
>
> tools/import_srcdist:
>      Check with "git status" that all files in the workspace have been
>      added, not ignored.
>
> diff --git a/tools/import_srcdist b/tools/import_srcdist
> index 51e861ba8..833d4acf0 100755
> --- a/tools/import_srcdist
> +++ b/tools/import_srcdist
> @@ -51,5 +51,10 @@ git checkout --detach "$initialcommit"
>   git clean -fdxq .
>   tar xf "$archive" --strip-components=1
>   git add -f .
> +if git status --porcelain=v1 --ignored | grep -v '^A  '
> +then
> +    echo "Some files were not added." >&2
> +    exit 1
> +fi
>   git commit -q -m "Imported $basename"
>   git tag -a -m "Tag $version" "$tag"