[m-rev.] for review: Disable tabling progs in parallel grades in tests/debugger.

[Zoltan Somogyi]

2022-01-22 11:59 GMT+11:00 "Peter Wang" <novalazy at gmail.com>: 
>> What convention would you propose? Simply a filename
>> containing the string "submodule"?
> 
> I suggest "_helper", which should be pretty obvious, or perhaps "_aux"
> for something shorter. Not all additional modules are submodules.
> I think we want to leave "submodule" for tests about submodules.

I agree: either _helper or _aux would be good, but we should support
only one of those, so that every module in test xyz that is not xyz.m itself
is caught by e.g. "xyz_aux*m".

>> >  2. Each test case FOO can be controlled by creating other files named
>> >     FOO.something. To disable 'loopcheck' in .par grades, there could be
>> >     a file 'loopcheck.disable' that contains something like:
>> > 
>> > 	.par
>> > 
>> >     which tells the test infrastructure to skip it in .par grades.
>> 
>> Agreed, though in the interest of keeping ls output easier to read,
>> I would use DISABLE/loopcheck instead.
>> 
> 
> I'd prefer to see everything related to foo with "ls foo*".
> If I want to see the list of test cases I'd run "ls *.m".

I can live with that.
 
> Compare that with the Prince test system, which I'm also familiar with.
> To run all tests in a directory, you run:
> 
>     ./runtests.sh
> 
> To run a subset of the tests, you'd run:
> 
>     ./runtests.sh foo*.html

I will look into doing something like that. I presume you would be
willing to review such changes?

> While we're on the subject, I'd also appreciate if the test system would
> be less noisy by default.

I just posted a diff for your review that reduces the size of bootcheck's
output by ~25%. What is left is mostly

- the commands building stages 2 and 3,
- one line for each successful test case, and
- a page or so for each unsuccessful test case.

I use the first two to judge how far an in-progress bootcheck has got,
which allows me to estimate when it will be done. To please both you and me,
we could get bootcheck to append short messages of the form "stage N task T done"
or "test dir TD done" to a file named perhaps .bootcheck progress.
We could then arrange for the output of building stages 2 and 3 to be put
into another file, to be consulted only when there is an error, or when there is
some uncertainty about whether the correct actions were executed. And we
could delete the messages about successful test cases.

As for the unsuccessful test cases, I find it useful for the output of bootcheck
to contain not just the list of unsuccessful test cases, but their logs too,
because it allows me to skip the extra step of opening those log files.
However, I see no reason why this behavior cannot be conditional,
so I can turn in on and you can turn it off. The main question is: turn in on/off
how? An option that has to be given to bootcheck on every invocation
is too invasive when you want the option's nondefault value. We *could*
set the default value based on the presence or absence, or the contents,
of a dot file either in the workspace, or in the executing user's home directory.
That would make the option "set-and-forget" either on a per-workspace
or a global basis.

Zoltan.