[m-rev.] for review: Disable tabling progs in parallel grades in tests/debugger.

[Peter Wang]

On Sun, 23 Jan 2022 21:56:05 +1100 "Zoltan Somogyi" <zoltan.somogyi at runbox.com> wrote:
> 
> 2022-01-22 11:59 GMT+11:00 "Peter Wang" <novalazy at gmail.com>: 
> >> What convention would you propose? Simply a filename
> >> containing the string "submodule"?
> > 
> > I suggest "_helper", which should be pretty obvious, or perhaps "_aux"
> > for something shorter. Not all additional modules are submodules.
> > I think we want to leave "submodule" for tests about submodules.
> 
> I agree: either _helper or _aux would be good, but we should support
> only one of those, so that every module in test xyz that is not xyz.m itself
> is caught by e.g. "xyz_aux*m".

Right.

> > Compare that with the Prince test system, which I'm also familiar with.
> > To run all tests in a directory, you run:
> > 
> >     ./runtests.sh
> > 
> > To run a subset of the tests, you'd run:
> > 
> >     ./runtests.sh foo*.html
> 
> I will look into doing something like that. I presume you would be
> willing to review such changes?
> 

Sure.

> > While we're on the subject, I'd also appreciate if the test system would
> > be less noisy by default.
> 
> I just posted a diff for your review that reduces the size of bootcheck's
> output by ~25%. What is left is mostly
> 
> - the commands building stages 2 and 3,
> - one line for each successful test case, and
> - a page or so for each unsuccessful test case.
> 
> I use the first two to judge how far an in-progress bootcheck has got,
> which allows me to estimate when it will be done. To please both you and me,
> we could get bootcheck to append short messages of the form "stage N task T done"
> or "test dir TD done" to a file named perhaps .bootcheck progress.
> We could then arrange for the output of building stages 2 and 3 to be put
> into another file, to be consulted only when there is an error, or when there is
> some uncertainty about whether the correct actions were executed. And we
> could delete the messages about successful test cases.

I always redirect bootchecks to a file and check progress with a script
using this regexp:

    regexp='^(building of.*|starting at.*|finishing at.*|FAILED TEST|PASSED ALL TESTS)'

so overall bootcheck output doesn't bother me. I bring up the test suite
output because it's useful to run some tests manually while developing a
feature or fix, and somehow I never remember to filter that output.

> As for the unsuccessful test cases, I find it useful for the output of bootcheck
> to contain not just the list of unsuccessful test cases, but their logs too,
> because it allows me to skip the extra step of opening those log files.
> However, I see no reason why this behavior cannot be conditional,
> so I can turn in on and you can turn it off. The main question is: turn in on/off
> how? An option that has to be given to bootcheck on every invocation
> is too invasive when you want the option's nondefault value. We *could*
> set the default value based on the presence or absence, or the contents,
> of a dot file either in the workspace, or in the executing user's home directory.
> That would make the option "set-and-forget" either on a per-workspace
> or a global basis.

An environment variable would work.

Peter