[m-rev.] for review: Disable tabling progs in parallel grades in tests/debugger.

[Peter Wang]

On Fri, 21 Jan 2022 17:17:10 +1100 "Zoltan Somogyi" <zoltan.somogyi at runbox.com> wrote:
> 
> 2022-01-21 17:10 GMT+11:00 "Peter Wang" <novalazy at gmail.com>:
> > tests/debugger/Mmakefile:
> >     As above. In particular, the loopcheck test would run indefinitely
> >     until killed, generating copious amount of output in the process.
> 
> That's fine, though I think we should decide whether we want
> to effect such decisions by overwriting the values of some
> make variables, like this:
> 
> > +ifneq "$(findstring par,$(GRADE))" ""
> > +    TABLING_PROGS :=
> > +endif
> 
> or whether we want to do a case analysis over the set of grades,
> and select which sets of test cases get included in PROGS,
> as in declarative_debugger/Mmakefile. Doing the same thing
> in different ways in different test directies seems inelegant.

The whole test infrastructure has always been horrible to work with.

If we were to start over, I think my suggestions would be:

 1. To create a test case, just add the .m file to the directory.
    Don't require it to be listed in a Mmakefile or whatever.
    We can establish a convention for .m files that aren't
    the top-level module of a test case.

 2. Each test case FOO can be controlled by creating other files named
    FOO.something. To disable 'loopcheck' in .par grades, there could be
    a file 'loopcheck.disable' that contains something like:

	.par

    which tells the test infrastructure to skip it in .par grades.

    Other things you might want to control are: how to invoke the test
    case, how to normalise the output, how to determine a passing test,
    etc.

  3. Make it easier to run individual test cases.

That's off the top of my head.

Peter