[m-rev.] for review: Grade java testing

Michael Wybrow mjwybrow at cs.mu.OZ.AU
Fri Jan 10 19:50:59 AEDT 2003
Previous message: [m-rev.] for review: fixes to extras/*
Next message: [m-rev.] for review: Grade java testing
Messages sorted by: [ date ] [ thread ] [ subject ] [ author ]
This was an outstanding patch I had from my studentship last summer.
The minor review comments from Fergus have been addressed and the patch
has been modified to work with the mmake test system.

I do have a couple of questions:
Why are some entire directories in the test suite disabled? (i.e. Why is
the recompilation directory not listed in `tests/Mmakefile'.)  If this is
intentional where is it documented?  Also, should I modify the Mmakefile
for such directories to disable their testing in grade java?

===================================================================

Estimated hours taken: 13
Branches: main

Modifications to the test suite to allow testing in grade java.


mercury/tools/test_mercury:
	Set-up the required CLASSPATH variable.

tests/Mmake.common:
	Added a rule to build `.out' files when the grade is java.

tests/benchmarks/Mmakefile:
	Enable only the working tests for the benckmarks directory.

tests/general/Mmakefile:
tests/general/accumulator/Mmakefile:
tests/general/string_format/Mmakefile:
tests/general/structure_reuse/Mmakefile:a
tests/grade_subdirs/Mmakefile:
tests/hard_coded/Mmakefile:
tests/hard_coded/exceptions/Mmakefile:
tests/hard_coded/purity/Mmakefile:
tests/hard_coded/sub-modules/Mmakefile:
tests/hard_coded/typeclasses/Mmakefile:
tests/term/Mmakefile:
tests/valid/Mmakefile:
	Disable test cases for grade java.

tests/general/accumulator/runtests:
	Workaround for a bug caused when we don't run any test cases in the
	general/accumulator directory.

tests/recompilation/test_functions:
	Add framework needed to do recompilation testing for the java grade.


Index: tests/Mmake.common
===================================================================
RCS file: /home/mercury1/repository/tests/Mmake.common,v
retrieving revision 1.37
diff -u -r1.37 Mmake.common
--- tests/Mmake.common	10 Jan 2003 05:19:03 -0000	1.37
+++ tests/Mmake.common	10 Jan 2003 06:22:19 -0000
@@ -49,6 +49,9 @@
 # deep profiling.
 RUN_RECOMPILATION_TESTS=yes
 
+# The Java interpreter.
+JAVA=java
+
 # We avoid picking up parameter settings from $HOME/.mdbrc that could
 # cause spurious differences between the outputs of the debugger test
 # cases and their expected outputs.
@@ -84,9 +87,19 @@
 # to print out the contents, because that precedes each line of output with
 # the filename, which is helpful when running a parallel make.
 #
+ifneq "$(findstring java,$(GRADE))" ""
+
+%.out: %.class
+	{ [ -f $*.inp ] && cat $*.inp; } | $(JAVA) $* > $@ 2>&1 \
+		|| { grep . $@ /dev/null; exit 1; }
+
+else
+
 %.out: %$(EXT_FOR_EXE)
 	{ [ -f $*.inp ] && cat $*.inp; } | ./$< > $@ 2>&1 || \
 		{ grep . $@ /dev/null; exit 1; }
+
+endif
 
 #
 # For some test cases, there is more than one valid output.
Index: tests/benchmarks/Mmakefile
===================================================================
RCS file: /home/mercury1/repository/tests/benchmarks/Mmakefile,v
retrieving revision 1.6
diff -u -r1.6 Mmakefile
--- tests/benchmarks/Mmakefile	12 Sep 2002 11:39:06 -0000	1.6
+++ tests/benchmarks/Mmakefile	8 Jan 2003 05:11:57 -0000
@@ -8,18 +8,27 @@
 
 #-----------------------------------------------------------------------------#
 
-PROGS=	\
+JAVA_PROGS= \
 	cqueens \
 	crypt \
 	deriv \
 	deriv2 \
 	nrev \
-	poly \
 	primes \
 	qsort \
 	queens \
 	query \
 	tak
+
+# XXX These test cases don't work in Java yet
+NONJAVA_PROGS= \
+	poly
+
+ifneq "$(findstring java,$(GRADE))" ""
+	PROGS=$(JAVA_PROGS)
+else
+	PROGS=$(JAVA_PROGS) $(NONJAVA_PROGS)
+endif
 
 TESTS=$(PROGS)
 SUBDIRS=
Index: tests/general/Mmakefile
===================================================================
RCS file: /home/mercury1/repository/tests/general/Mmakefile,v
retrieving revision 1.49
diff -u -r1.49 Mmakefile
--- tests/general/Mmakefile	28 Nov 2002 09:13:35 -0000	1.49
+++ tests/general/Mmakefile	10 Jan 2003 07:28:33 -0000
@@ -71,10 +71,14 @@
 		map_corresponding \
 		unused_arg_determinism
 
-ifneq "$(findstring profdeep,$(GRADE))" ""
-	PROGS0 = $(ORDINARY_PROGS)
+ifneq "$(findstring java,$(GRADE))" ""
+	PROGS0 =
 else
-	PROGS0 = $(ORDINARY_PROGS) $(EXCEPTION_PROGS)
+	ifneq "$(findstring profdeep,$(GRADE))" ""
+		PROGS0 = $(ORDINARY_PROGS)
+	else
+		PROGS0 = $(ORDINARY_PROGS) $(EXCEPTION_PROGS)
+	endif
 endif
 
 	# On Dec OSF 5.1 the floating point tests of
Index: tests/general/accumulator/Mmakefile
===================================================================
RCS file: /home/mercury1/repository/tests/general/accumulator/Mmakefile,v
retrieving revision 1.18
diff -u -r1.18 Mmakefile
--- tests/general/accumulator/Mmakefile	17 Aug 2002 13:52:07 -0000	1.18
+++ tests/general/accumulator/Mmakefile	18 Dec 2002 00:58:24 -0000
@@ -6,7 +6,8 @@
 
 # Any program added here should also be added to the `.cvsignore' file.
 
-PROGS=		base		\
+ACCUMULATOR_PROGS= \
+		base		\
 		call_in_base	\
 		chain		\
 		commutative	\
@@ -26,6 +27,14 @@
 		simple		\
 		split		\
 		swap
+
+# We currently don't do any testing in grade java on this directory.
+ifneq "$(findstring java,$(GRADE))" ""
+	PROGS=
+else
+	PROGS=$(ACCUMULATOR_PROGS)
+endif
+
 
 SUBDIRS=
 TESTS = $(PROGS)
Index: tests/general/string_format/Mmakefile
===================================================================
RCS file: /home/mercury1/repository/tests/general/string_format/Mmakefile,v
retrieving revision 1.9
diff -u -r1.9 Mmakefile
--- tests/general/string_format/Mmakefile	5 Dec 2002 08:45:20 -0000	1.9
+++ tests/general/string_format/Mmakefile	10 Jan 2003 07:17:30 -0000
@@ -22,7 +22,12 @@
 ifneq "$(findstring .agc,$(GRADE))" ""
 	PROGS0=
 else
-	PROGS0=$(STRING_FORMAT_PROGS)
+	# We currently don't do any testing in grade java on this directory.
+	ifneq "$(findstring java,$(GRADE))" ""
+		PROGS0=
+	else
+		PROGS0=$(STRING_FORMAT_PROGS)
+	endif
 endif
 
 	# On solaris 5.{7,8} string_format_o fails because of a buggy
Index: tests/general/structure_reuse/Mmakefile
===================================================================
RCS file: /home/mercury1/repository/tests/general/structure_reuse/Mmakefile,v
retrieving revision 1.3
diff -u -r1.3 Mmakefile
--- tests/general/structure_reuse/Mmakefile	17 Aug 2002 13:52:09 -0000	1.3
+++ tests/general/structure_reuse/Mmakefile	18 Dec 2002 01:01:38 -0000
@@ -4,9 +4,17 @@
 
 # Any program added here should also be added to the `.cvsignore' file.
 
-PROGS=	if_then_else		\
+STRUCTURE_REUSE_PROGS=		\
+	if_then_else		\
 	internal_alias		\
 	interpret
+
+# We currently don't do any testing in grade java on this directory.
+ifneq "$(findstring java,$(GRADE))" ""
+	PROGS=
+else
+	PROGS=$(STRUCTURE_REUSE_PROGS)
+endif
 
 TESTS=$(PROGS)
 TESTS_DIR=../..
Index: tests/grade_subdirs/Mmakefile
===================================================================
RCS file: /home/mercury1/repository/tests/grade_subdirs/Mmakefile,v
retrieving revision 1.2
diff -u -r1.2 Mmakefile
--- tests/grade_subdirs/Mmakefile	30 Nov 2002 14:32:44 -0000	1.2
+++ tests/grade_subdirs/Mmakefile	10 Jan 2003 06:50:11 -0000
@@ -3,13 +3,18 @@
 
 THIS_DIR = grade_subdirs
 
-ifndef WORKSPACE
-PROGS=\
-	hello
+# We currently don't do any testing in grade java on this directory.
+ifneq "$(findstring java,$(GRADE))" ""
+	PROGS=
 else
-# We can only use a workspace if it was built with `--use-grade-subdirs',
-# which in general it won't be.
-PROGS=
+	ifndef WORKSPACE
+		PROGS=\
+			hello
+	else
+		# We can only use a workspace if it was built with
+		# `--use-grade-subdirs', which in general it won't be.
+		PROGS=
+	endif
 endif
 
 # `--use-grade-subdirs' doesn't work with Mmake.
Index: tests/hard_coded/Mmakefile
===================================================================
RCS file: /home/mercury1/repository/tests/hard_coded/Mmakefile,v
retrieving revision 1.179
diff -u -r1.179 Mmakefile
--- tests/hard_coded/Mmakefile	4 Dec 2002 01:57:29 -0000	1.179
+++ tests/hard_coded/Mmakefile	10 Jan 2003 01:30:06 -0000
@@ -283,9 +283,16 @@
 	endif
 endif
 
-PROGS = $(ORDINARY_PROGS) $(BROKEN_FOR_LCC_PROGS) $(CLOSURE_LAYOUT_PROGS) \
-	$(EXCEPTION_PROGS) $(BACKEND_PROGS) $(NONDET_C_PROGS) \
-	$(C_AND_GC_ONLY_PROGS) $(STATIC_LINK_PROGS) $(CHAR_REP_PROGS)
+# We currently don't do any testing in grade java on this directory.
+ifneq "$(findstring java,$(GRADE))" ""
+	PROGS =
+else
+	PROGS = $(ORDINARY_PROGS) $(BROKEN_FOR_LCC_PROGS) \
+		$(CLOSURE_LAYOUT_PROGS) $(EXCEPTION_PROGS) \
+		$(BACKEND_PROGS) $(NONDET_C_PROGS) \
+		$(C_AND_GC_ONLY_PROGS) $(STATIC_LINK_PROGS) \
+		$(CHAR_REP_PROGS)
+endif
 
 # --split-c-files does not work in the hl* grades (e.g. hlc.gc),
 # because it hasn't yet been implemented yet.
Index: tests/hard_coded/exceptions/Mmakefile
===================================================================
RCS file: /home/mercury1/repository/tests/hard_coded/exceptions/Mmakefile,v
retrieving revision 1.10
diff -u -r1.10 Mmakefile
--- tests/hard_coded/exceptions/Mmakefile	17 Aug 2002 13:52:14 -0000	1.10
+++ tests/hard_coded/exceptions/Mmakefile	17 Dec 2002 05:18:06 -0000
@@ -35,8 +35,9 @@
 
 # Deep profiling grades cannot yet handle catching exceptions, either
 # explicitly or implicitly by the runtime system.
+# We currently don't do any testing in grade java on this directory.
 
-ifneq "$(findstring profdeep,$(GRADE))" ""
+ifneq "$(findstring profdeep,$(GRADE))$(findstring java,$(GRADE))" ""
 	PROGS=
 else
 	PROGS=$(EXCEPTION_PROGS)
Index: tests/hard_coded/purity/Mmakefile
===================================================================
RCS file: /home/mercury1/repository/tests/hard_coded/purity/Mmakefile,v
retrieving revision 1.2
diff -u -r1.2 Mmakefile
--- tests/hard_coded/purity/Mmakefile	17 Aug 2002 13:52:15 -0000	1.2
+++ tests/hard_coded/purity/Mmakefile	18 Dec 2002 01:06:13 -0000
@@ -4,10 +4,17 @@
 
 #-----------------------------------------------------------------------------#
 
-PROGS=	\
+PURITY_PROGS= \
 	purity \
 	impure_func_t1 \
 	impure_func_t6 
+
+# We currently don't do any testing in grade java on this directory.
+ifneq "$(findstring java,$(GRADE))" ""
+	PROGS=
+else
+	PROGS=$(PURITY_PROGS)
+endif
 
 TESTS = $(PROGS)
 SUBDIRS=
Index: tests/hard_coded/sub-modules/Mmakefile
===================================================================
RCS file: /home/mercury1/repository/tests/hard_coded/sub-modules/Mmakefile,v
retrieving revision 1.7
diff -u -r1.7 Mmakefile
--- tests/hard_coded/sub-modules/Mmakefile	17 Aug 2002 13:52:16 -0000	1.7
+++ tests/hard_coded/sub-modules/Mmakefile	18 Dec 2002 01:07:31 -0000
@@ -14,7 +14,7 @@
 # if you use parallel make.
 #
 
-PROGS=	\
+SUB_MODULE_PROGS= \
 	accessibility \
 	accessibility2 \
 	use_submodule \
@@ -25,6 +25,13 @@
 	nested3 \
 	class \
 	nested_intermod_main
+
+# We currently don't do any testing in grade java on this directory.
+ifneq "$(findstring java,$(GRADE))" ""
+	PROGS=
+else
+	PROGS=$(SUB_MODULE_PROGS)
+endif
 
 TESTS = $(PROGS)
 TESTS_DIR=../..
Index: tests/hard_coded/typeclasses/Mmakefile
===================================================================
RCS file: /home/mercury1/repository/tests/hard_coded/typeclasses/Mmakefile,v
retrieving revision 1.50
diff -u -r1.50 Mmakefile
--- tests/hard_coded/typeclasses/Mmakefile	22 Oct 2002 14:13:22 -0000	1.50
+++ tests/hard_coded/typeclasses/Mmakefile	10 Jan 2003 06:28:43 -0000
@@ -4,7 +4,7 @@
 
 #-----------------------------------------------------------------------------#
 
-PROGS=	\
+TYPECLASSES_PROGS= \
 	arbitrary_constraint_class \
 	arbitrary_constraint_pred_2 \
 	arbitrary_constraint_pred_1 \
@@ -64,6 +64,13 @@
 	unqualified_method \
 	use_abstract_instance \
 	use_abstract_typeclass
+
+# We currently don't do any testing in grade java on this directory.
+ifneq "$(findstring java,$(GRADE))" ""
+	PROGS=
+else
+	PROGS=$(TYPECLASSES_PROGS)
+endif
 
 TESTS = $(PROGS)
 SUBDIRS=
Index: tests/recompilation/test_functions
===================================================================
RCS file: /home/mercury1/repository/tests/recompilation/test_functions,v
retrieving revision 1.6
diff -u -r1.6 test_functions
--- tests/recompilation/test_functions	17 Aug 2002 13:52:25 -0000	1.6
+++ tests/recompilation/test_functions	18 Dec 2002 01:36:19 -0000
@@ -128,6 +128,17 @@
 	output_file_version=$1
 	mmake_should_fail=$2
 
+	# XXX This is a hacky way of testing for Java grades
+	echo "mmakeopts=$mmakeopts"
+	case $mmakeopts in
+		*java*)	target=$main_module.classes
+			run_target="java $main_module"
+			;;
+		*)	target=$main_module
+			run_target="./$main_module"
+			;;
+	esac
+
 	case $mmake_should_fail in
 	    true)	
 		#
@@ -138,10 +149,10 @@
 		# when using parallel mmakes.
 		#
 		eval mmake $mmakeopts -k \
-			$main_module > $main_module.failing_make_output 2>&1
+			$target > $main_module.failing_make_output 2>&1
 		;;
 	    false)
-		eval mmake $mmakeopts $main_module
+		eval mmake $mmakeopts $target
 		;;
 	esac
 
@@ -150,19 +161,19 @@
 		case $mmake_should_fail in
 		    true)
 			echo \
-	"** Error: mmake $mmakeopts $main_module succeeded where it should fail"
+	"** Error: mmake $mmakeopts $target succeeded where it should fail"
 			exit 1
 			;;
 		esac
 
-		./$main_module > $main_module.out
+		$run_target > $main_module.out
 		compare_files $main_module.exp.$output_file_version \
 				$main_module.out 
 		;;
 	    *)
 		case $mmake_should_fail in
 		    false)
-			echo "** Error: mmake $mmakeopts $main_module failed"
+			echo "** Error: mmake $mmakeopts $target failed"
 			exit 1
 			;;
 		esac
Index: tests/term/Mmakefile
===================================================================
RCS file: /home/mercury1/repository/tests/term/Mmakefile,v
retrieving revision 1.18
diff -u -r1.18 Mmakefile
--- tests/term/Mmakefile	20 Sep 2002 09:40:38 -0000	1.18
+++ tests/term/Mmakefile	18 Dec 2002 01:38:06 -0000
@@ -6,9 +6,9 @@
 
 ifeq ($(MMAKE_USE_MMC_MAKE),yes)
 # `mmc --make' doesn't know how to create `.trans_opt' files.
-PROGS=
+TERM_PROGS=
 else
-PROGS=	\
+TERM_PROGS= \
 	ack \
 	append \
 	arit_exp \
@@ -73,6 +73,13 @@
 # Module-specific options should go in Mercury.options so they
 # can be found by `mmc --make'.
 include Mercury.options
+
+# We currently don't do any testing in grade java on this directory.
+ifneq "$(findstring java,$(GRADE))" ""
+	PROGS=
+else
+	PROGS=$(TERM_PROGS)
+endif
 
 %.runtest: %.trans_opt_res ;
 
Index: tests/valid/Mmakefile
===================================================================
RCS file: /home/mercury1/repository/tests/valid/Mmakefile,v
retrieving revision 1.119
diff -u -r1.119 Mmakefile
--- tests/valid/Mmakefile	22 Nov 2002 08:50:43 -0000	1.119
+++ tests/valid/Mmakefile	18 Dec 2002 01:22:22 -0000
@@ -233,12 +233,17 @@
 # (i.e. grades hl* java* il*).
 # It will never be implemented for deep profiling grades.
 ifneq "$(filter hl% java% il%,$(GRADE))$(findstring profdeep,$(GRADE))" ""
-    OBJ_PROGS=$(PROGS2)
+    # We currently don't do any testing in grade java on this directory.
+    ifneq "$(findstring java,$(GRADE))$" ""
+        OBJ_PROGS=
+    else
+        OBJ_PROGS=$(PROGS2)
+    endif
 else
     OBJ_PROGS=$(PROGS2) $(ADITI_PROGS)
 endif
 
-ifneq "$(findstring profdeep,$(GRADE))" ""
+ifneq "$(findstring profdeep,$(GRADE))$(findstring java,$(GRADE))" ""
 	ALL_RLO_PROGS =
 else
 	ALL_RLO_PROGS = $(RLO_PROGS)
Index: mercury/tools/test_mercury
===================================================================
RCS file: /home/mercury1/repository/mercury/tools/test_mercury,v
retrieving revision 1.226
diff -u -r1.226 test_mercury
--- mercury/tools/test_mercury	2 Jan 2003 06:54:03 -0000	1.226
+++ mercury/tools/test_mercury	8 Jan 2003 02:26:56 -0000
@@ -70,6 +70,10 @@
 HOSTDIR=$TESTDIR/test_dirs/$HOST
 DIR=$HOSTDIR/build.$BRANCH.$C_COMPILER
 
+# Set-up the class path needed for java testing.
+CLASSPATH=$DIR/mercury/java:.
+export CLASSPATH
+
 # Host to use to build the release of the day source distribution
 # for the mercury-compiler, mercury-tests, and mercury-extras distributions.
 ROTD_HOST=earth

--------------------------------------------------------------------------
mercury-reviews mailing list
post:  mercury-reviews at cs.mu.oz.au
administrative address: owner-mercury-reviews at cs.mu.oz.au
unsubscribe: Address: mercury-reviews-request at cs.mu.oz.au Message: unsubscribe
subscribe:   Address: mercury-reviews-request at cs.mu.oz.au Message: subscribe
--------------------------------------------------------------------------
Previous message: [m-rev.] for review: fixes to extras/*
Next message: [m-rev.] for review: Grade java testing
Messages sorted by: [ date ] [ thread ] [ subject ] [ author ]
More information about the reviews mailing list