[m-dev.] .NET back-end test results

[Fergus Henderson]

On 27-Dec-2002, Peter Ross <pro at missioncriticalit.com> wrote:
> Date            Number of test case failures
> 20 Oct 2002     459
> 07 Nov 2002     202
> 28 Nov 2002     73
> 05 Dec 2002     27
> 27 Dec 2002     21

Since then we have fixed three of those test case failures,
which you might think would reduce the number of failures to 18.
But nevertheless, I now get the following results:

  Date            Number of test case failures
  Feb 25 2003     58

This mainly because we have added or enabled 36 new test cases --
including some which were present but which were not previously being run
(dppd/run.m and recompilation/*) -- that happen to fail in grade `il'.

But I also got two new failures that showed up because I ran these
tests with a version of the compiler built in grade `il'.
And there are also two new spurious failures in tests/invalid;
I'm not sure why these didn't fail previously.

Anyway, here's my latest summary of test results for grade `il'.

SUMMARY:

	CAUSE			NUMBER OF FAILING TESTS
	Bugs:			 2
	Minor bugs:		 4		
	NYI required feature:	40

	NYI optional feature:	 6	(tests should be disabled for `il')
	Spurious failures:	 5	(bugs in the test cases)
	Unreproducable:	 	 1


BUGS:
- hard_coded/null_char:
	BUG: strings containing special characters are not properly
	escaped in the generated IL code. 
	[New failure.  This bug won't show up for this test case when
	cross-compiling from C grades.]

- hard_coded/exceptions/test_try_all:
	BUG: MissingMethodException for Void mercury.exception__cpp_code.
	mercury_code.builtin_catch_3_m4(System.Object[], System.Object[],
	System.Object[], Int32, UIntPtr).

MINOR BUGS:
- general/string_to_float: MINOR BUG: -0.0 output as 0.0
- hard_coded/sub-modules/parent2:
	MINOR BUG: .NET back-end requires main/2 be defined in top-level module
- hard_coded/sub-modules/nested2:
	MINOR BUG: .NET back-end requires main/2 be defined in top-level module
- invalid/uniq_neg-nodepend in grade il
	MINOR BUG: spurious warning
	store.int:020: In `set_mutvar(in, in, di, uo)':
	store.int:020:   warning: determinism declaration could be tighter.
	store.int:020:   Declared `det', inferred `erroneous'.
	caused by store.m being NYI

NOT YET IMPLEMENTED (feature required as part of standard Mercury)
- dppd/run: NYI: benchmarking.m [new test]
- hard_coded/construct_test: NYI: construct
- hard_coded/higher_order_type_manip: NYI: type_ctor
- hard_coded/nullary_ho_func:
	NYI: RTTI for higher-order function types returns arity = -1
- hard_coded/time_test: NYI: time.m
- hard_coded/tuple_test: NYI construct
- hard_coded/type_ctor_desc:
	NYI: RTTI for higher-order function types returns arity = -1
- hard_coded/type_ctor_desc_manip: NYI type_ctor
- hard_coded/exceptions/tricky_try_store:
	NYI: store__do_init
- valid/aditi_query:
	NYI: Sorry, unimplemented: foreign code for this function, open mode:wb
	[New failure.  Shows up when writing the Aditi bytecode, if the
	compiler was built in grade `il', because io__open_binary_output_stream
	is NYI.]
- recompilation/* (30 test cases): NYI: time.m [new tests]

NOT YET IMPLEMENTED OPTIONAL (feature not part of standard Mercury):
- hard_coded/pragma_export: NYI OPTIONAL: pragma import
- hard_coded/pragma_import: NYI OPTIONAL: pragma import
- valid/intermod_pragma_import: NYI OPTIONAL: pragma import
- hard_coded/factt: NYI OPTIONAL: fact tables
- invalid/reserve_tag-nodepend in grade il
	NYI OPTIONAL: pragma reserve_tag (no primary tags available!)
	[new test]
- valid/reserve_tag:
	NYI OPTIONAL: pragma reserve_tag (no primary tags available!)
	[new test]

SPURIOUS FAILURES:
- hard_coded/string_strip:
	[new test]
	SPURIOUS failure due to different string ordering?
- hard_coded/user_compare:
	SPURIOUS failure due to missing IL/C# foreign code in test case
	[new test]
- invalid/foreign_type_2-nodepend:
	[regression?]
	SPURIOUS: error reported when making `.opt' file.
- invalid/merge_ground_any-nodepend in grade il
	[regression?]
	SPURIOUS: error reported when making `.opt' file.
- warnings/singleton_test:
	SPURIOUS: test case missing IL/C# foreign code in test case

UNKNOWN:
- warnings/up_to_date:
	[new test]
	XXX failure cause unknown, can't reproduce;
	some kind of transient failure?

> grade_subdirs/hello
>     we attempt to make libhello.a, which doesn't make sense for the il
>     backend.
...
> hard_coded/type_qual
>     requires more than one character of putback
...
> valid/soln_context
>     not sure why this is an error, needs more investigation

These three are fixed now.

-- 
Fergus Henderson <fjh at cs.mu.oz.au>  |  "I have always known that the pursuit
The University of Melbourne         |  of excellence is a lethal habit"
WWW: <http://www.cs.mu.oz.au/~fjh>  |     -- the last words of T. S. Garp.
--------------------------------------------------------------------------
mercury-developers mailing list
Post messages to:       mercury-developers at cs.mu.oz.au
Administrative Queries: owner-mercury-developers at cs.mu.oz.au
Subscriptions:          mercury-developers-request at cs.mu.oz.au
--------------------------------------------------------------------------