[m-dev.] .NET back-end test results
fjh at cs.mu.OZ.AU
Sun Oct 20 09:38:49 AEST 2002
I've run the tests in the `il' grade and analyzed the results.
Here are the results of my analysis.
First, the number of distinct bug symptoms detected:
Bugs in the test suite: 5
Bugs in the .NET back-end: 8
Unimplemented features: 7
So, at one patch per bug symptom or unimplemented feature,
we are just 20 patches away from passing all the tests ;-)
Of course, it may be the case that the same symptoms are being
generated by more than one bug, in which case more bug-fixes
will be needed. Also, there may be some additional bugs which
show up once we've fixed all of those.
Second, the number of test case failures.
Note that each bug can in general cause many test case failures.
Passed test cases: 280
Failed test cases: 459 total
Spurious failures: 291 subtotal
Failures as a result of genuine bugs: 117 subtotal
Failures due to features that are not yet implemented,
for which we output an appropriate "sorry" message: 51
Inclued below is a detailed analysis.
- test/valid/* report spurious failures due to Mmakefile making
incorrect assumption that we generate `.o' files (FAILED.VALID)
- test/invalid/* report spurious failures due to the Mmakefile not catering
for --il implying --intermodule-optimization (FAILED.INVALID)
- lots of tests fail due to the use of C foreign clauses (FAILED.NO_CLAUSES)
tests/general/impure_foreign is a special case of this;
it fails at run-time because the Mercury fall-back clauses just call error/1.
tests/hard_coded/pragma_import and tests/hard_coded/pragma_export are
- tests/hard_coded/*.split (not supported for il grade)
- tests/hard_coded/special_character fails, due to '\0321' in the
source code getting converted to octal 245 in the output file.
This is a bug in the test case; it assumes that the internal
representation is the same as the file representation, which is
not the case for .NET.
- tests/hard_coded/term_io_test fails:
gets syntax errors at run-time, cause unknown.
- tests/general/commutative fails:
Likewise for lots of others (FAILED.INVALID_CAST)
- tests/general/interpreter fails, due to problem with equivalence types;
the argument types for the unify & compare predicates for equivalence types
are invalid (undefined)
Likewise for lots of others (FAILED.UNDEFINED_CLASS)
- tests/general/string_to_float fails,
because string__to_float throws an exception rather than failing
- tests/general/string_format_test fails, due to various formatting errors
(e.g. no left justification, missing 0x prefixes, not truncating).
- tests/hard_coded/*foreign_type* fail, due to not finding `mercury.dll'.
(XXX probably due to me not installing it properly)
- tests/hard_coded/no_fully_strict fails,
because uncaught exceptions do not result in a non-zero exit status
- tests/hard_coded/no_warn_singleton fails,
due to determinism warning for table_io_copy_io_state from table_builtin.m
NOT YET IMPLEMENTED (FAILED.SORRY):
- lots of tests fail due to
- tests/general/double_match fails:
- tests/general/environment fails:
- io__read_line_as_string_2 NYI
- tests/general/map_corresponding fails:
Software Error: rtti_implementation: unimplemented: tuple comparisons
Likewise for tests/hard_coded/existential_tuples
- tests/general/io_regression fails:
- tests/hard_coded/random_permutation random_simple fail:
MissingMethodException due to copy/2 not being implemented
mercury-developers mailing list
Post messages to: mercury-developers at cs.mu.oz.au
Administrative Queries: owner-mercury-developers at cs.mu.oz.au
Subscriptions: mercury-developers-request at cs.mu.oz.au
More information about the developers