[m-dev.] Test cases (was: for review: big ints)

[Andrew Bromage]

G'day.

Tyson Dowd wrote:

> The tests directory is getting pretty full.  It's sometimes difficult
> to tell feature tests from regression tests, and lots of the feature
> tests are together. 

As a matter of interest, what does this buy you apart from general
cleanliness and good karma?

In one sense, I agree with you.  Knowing what the test was testing
for can help you diagnose the cause.  The problem is that last
year's regression tests for the code generator often double as
this year's regression tests for some semantic pass (a pathological
case is a pathological case), so such a neat filing system seems to
me unhelpful in the general case.

I think what would make a bigger difference is to discipline developers
to attach to test cases a summary of the log message when a bug is
fixed.  This would be more helpful information, because if a test fails,
what you have is some indication of where to look.

For example, while this might be useful information:

	tests/language/regression/lambda_mode_check_bug.m

I'd much rather see:

	% Non-local variables of lambdas were not mode checked
	% correctly by the 8 May 1997 compiler.  Problem was a
	% bug in modecheck_unification where the non-locals were
	% not computed correctly.
	%			- ahacker (9 May 1997)
	%
	% This test failed 2 Feb 1998 with a run-time abort
	% "variable P (8) not found" at opt level -O3.  The lambda
	% goal was not correctly quantified after eliminating the
	% excess assignment during simplify2.
	%			- anotherhacker (3 Feb 1998)

[Disclaimer: The above bugs are made up.  Any resemblance to real
bugs living or dead is purely coincidental.]

Cheers,
Andrew Bromage