[m-rev.] for review: improve test framework

[Zoltan Somogyi]

On 16-Aug-2002, Mark Brown <dougl at cs.mu.OZ.AU> wrote:
> It would be logical and useful to put those in a sub-directory.  (And, yes,
> if this were done you would want people who run tests in hard_coded to
> also run tests in hard_coded/commits.)

That just begs the question of how deep the hierarchy should be, and what the
criteria for choosing where to put a test should be.

What happens if I have the following four tests?

- one testing the static semantic analysis of code with switches
- one testing the static semantic analysis of code with commits
- one testing code generation for code with switches
- one testing code generation for code with commits

> And if Mmakefile programming is used, the
> "meaningful subsets" of test cases don't even have to form a hierachy;
> my only requirement would be that they have some valid rationale to
> back them up.

Sure. The question is whether the benefit would exceed the cost of programming
this. The fact that noone has done this so far argues that it may not.

> I'd be inclined to argue that there is a dependency there.  Insofar as
> the hard_coded test cases test the operation of complete Mercury
> programs,

It does, but almost all the other test directories (except invalid and
warnings) also test whole programs. tests/hard_coded/*.m are not special
in this.

Hard_coded's original defining characteristic, that it tests Mercury features
that don't exist in NU-Prolog, is long obsolete. Now, a test goes in hard_coded
if (a) its compilation is expected to succeed, and (b) it doesn't fit well
anywhere else. That is not a cohesive collection.

> Even if there was no dependency, I still don't see the rationale for
> testing all of the hard_coded tests, but nothing else.

For the last several iterations in this discussion, since Simon made his
suggestion, nobody has been arguing for this.

Zoltan.
--------------------------------------------------------------------------
mercury-reviews mailing list
post:  mercury-reviews at cs.mu.oz.au
administrative address: owner-mercury-reviews at cs.mu.oz.au
unsubscribe: Address: mercury-reviews-request at cs.mu.oz.au Message: unsubscribe
subscribe:   Address: mercury-reviews-request at cs.mu.oz.au Message: subscribe
--------------------------------------------------------------------------