[m-rev.] for review: improve test framework

[Zoltan Somogyi]

On 15-Aug-2002, Fergus Henderson <fjh at cs.mu.OZ.AU> wrote:
> > In think one of the problems with the current system is that you can't run the
> > tests in e.g. the hard_coded directory without also running the tests in the
> > hard_coded/exceptions, hard_coded/purity, hard_coded/sub-modules and
> > hard_coded/typeclasses directories.
> 
> Sure you can.
> 
> 	cd hard_coded
> 	mmake depend_local
> 	mmake check_local

I meant from bootcheck. The three lines above don't set up the environment
variables right. A tool separate from bootcheck could do that, but that would
be double maintenance. (Bootcheck can't call the tool because child processes
cannot affect the parent's environment. It could source a shell code fragment,
I suppose.)

When fixing bugs, I often use .KEEP_OBJS to keep all the object files in
stage2. If the fix passes the comparison of stage2 & stage3 but fails some
test cases, I can modify a file to attempt to fix the problem and remake stage2
(which often requires recompiling only a few files). I would then like to check
whether this fixes the test case failures.

Typically I test some by hand, but this can get tedious, so I then invoke
"bootcheck -2 -d dirname1 -d dirname2" to check the rest. Any time bootcheck
spends on subdirectories of the named directories is a waste of time,
because if the change fixes all the test cases, I of course also do a full
bootcheck from scratch to check for any new problems.

> > I think it would be simpler as well as more flexible if we listed all the
> > directories containing test cases in the top level tests directory,
> > regardless of whether they are direct or indirect subdirectories of the tests
> > directory.
> 
> That would mean there was no way to say "run all the tests in this
> directory and any of its subdirectories".

Two different options in bootcheck, one saying "execute the tests in this
directory only" and one saying "execute the tests in this directory and its
subdirectories" should be easy enough to arrange, and so would "mmake
check_local_and_subdirs".

Zoltan.
--------------------------------------------------------------------------
mercury-reviews mailing list
post:  mercury-reviews at cs.mu.oz.au
administrative address: owner-mercury-reviews at cs.mu.oz.au
unsubscribe: Address: mercury-reviews-request at cs.mu.oz.au Message: unsubscribe
subscribe:   Address: mercury-reviews-request at cs.mu.oz.au Message: subscribe
--------------------------------------------------------------------------