[m-rev.] for review: Disable tabling progs in parallel grades in tests/debugger.

[Zoltan Somogyi]

2022-01-21 17:55 GMT+11:00 "Peter Wang" <novalazy at gmail.com>:
> The whole test infrastructure has always been horrible to work with.

I wouldn't call it horrible, but I agree, it is "highly suboptimal".

> If we were to start over,

I think we should, after the release, though we should keep
the pieces that do work.

> I think my suggestions would be:
> 
>  1. To create a test case, just add the .m file to the directory.
>     Don't require it to be listed in a Mmakefile or whatever.
>     We can establish a convention for .m files that aren't
>     the top-level module of a test case.

What convention would you propose? Simply a filename
containing the string "submodule"?

>  2. Each test case FOO can be controlled by creating other files named
>     FOO.something. To disable 'loopcheck' in .par grades, there could be
>     a file 'loopcheck.disable' that contains something like:
> 
> 	.par
> 
>     which tells the test infrastructure to skip it in .par grades.

Agreed, though in the interest of keeping ls output easier to read,
I would use DISABLE/loopcheck instead.

>     Other things you might want to control are: how to invoke the test
>     case, how to normalise the output, how to determine a passing test,

Agreed.

>   3. Make it easier to run individual test cases.

For single-module test cases, and if you don't need to use mmc --make,
lmc works fine for me in most cases. I would love to lift both of those restrictions.
I have an idea for lifting the former, but don't know enough mmc --make
to lift the latter.

Zoltan.