[m-rev.] for post-commit review: fix deep profiling bootstrap bug

Zoltan Somogyi zs at csse.unimelb.edu.au
Mon Dec 24 10:56:27 AEDT 2007
Previous message: [m-rev.] for post-commit review: clean up ssdb.m
Next message: [m-rev.] for post-commit review: fix deep profiling bootstrap bug
Messages sorted by: [ date ] [ thread ] [ subject ] [ author ]
Fix a bug that prevented the compiler from bootstrapping in deep profiling
grades. The symptom was the violation of a sanity check in
mercury_deep_profiling.c, which lead to this message:

	Mercury runtime: UNWRITTEN nodes: Deep.data file corrupted

Mmake.workspace:
	Don't include the ssdb library's init file unconditionally in every
	_init.c file we generate; instead include the ssdb library's init file
	only in ssdb grades.

	The reason why this is needed is that the referenced but unwritten
	nodes were the proc layouts of three predicates in the browser
	directory. They were not written because the browser library's init
	file is (correctly) included in the compiler's _init.c file only if
	the browser library is linked into the compiler, i.e. only if the
	compiler is compiled in a debug grade. They were referenced by call
	site static structures from ssdb/ssdb.m, whose init file was
	(incorrectly) being included in the compiler's _init.c file even if
	the compiler wasn't being compiled in an ssdb grade.

scripts/parse_ml_options.sh-subr.in:
	Add the option needed by Mmake.workspace to add an init file only in an
	ssdb grade: --ssdb-init-file.

scripts/c2init.in:
	Act on the value of that option.

	Delete a reference to $TRACE_INIT_FILES, since it should always be
	empty. Document a similar problem with $EXTRA_INIT_FILES.

Mmakefile:
	Improve formatting.

compiler/layout_out.m:
	Avoid inconsistent indentation in the generated .c code.

tools/bootcheck:
	Add two options that helped track down the bug: --deep-all-write,
	which causes bootcheck to write out *every*deep profiling data file,
	and --deep-debug, which switches on the already existing deep profiling
	debug support.

runtime/mercury_deep_profiling.c:
	Put the two lines of the error message we generate for unwritten nodes
	if deep profiling's debug support is enabled in the logical order.

cvs diff: Diffing .
Index: Mmake.workspace
===================================================================
RCS file: /home/mercury/mercury1/repository/mercury/Mmake.workspace,v
retrieving revision 1.28
diff -u -b -r1.28 Mmake.workspace
--- Mmake.workspace	24 Oct 2007 09:21:13 -0000	1.28
+++ Mmake.workspace	23 Dec 2007 01:38:47 -0000
@@ -123,9 +123,9 @@
 # XXX browser.init and mdbcomp.init might need to be included even if tracing
 # is disabled if ssdb uses code from the browser or mdbcomp libraries
 C2INITFLAGS  += --trace-init-file $(BROWSER_DIR)/$(BROWSER_LIB_NAME).init \
-		--trace-init-file $(MDBCOMP_DIR)/$(MDBCOMP_LIB_NAME).init
-C2INITARGS   += $(SSDB_DIR)/$(SSDB_LIB_NAME).init \
-		$(LIBRARY_DIR)/$(STD_LIB_NAME).init \
+		--trace-init-file $(MDBCOMP_DIR)/$(MDBCOMP_LIB_NAME).init \
+		--ssdb-init-file $(SSDB_DIR)/$(SSDB_LIB_NAME).init
+C2INITARGS   += $(LIBRARY_DIR)/$(STD_LIB_NAME).init \
 		$(RUNTIME_DIR)/$(RT_LIB_NAME).init
 MLFLAGS      += --no-mercury-stdlib-dir
 
cvs diff: Diffing analysis
cvs diff: Diffing bindist
cvs diff: Diffing boehm_gc
cvs diff: Diffing boehm_gc/Mac_files
cvs diff: Diffing boehm_gc/cord
cvs diff: Diffing boehm_gc/cord/private
cvs diff: Diffing boehm_gc/doc
cvs diff: Diffing boehm_gc/include
cvs diff: Diffing boehm_gc/include/private
cvs diff: Diffing boehm_gc/libatomic_ops-1.2
cvs diff: Diffing boehm_gc/libatomic_ops-1.2/doc
cvs diff: Diffing boehm_gc/libatomic_ops-1.2/src
cvs diff: Diffing boehm_gc/libatomic_ops-1.2/src/atomic_ops
cvs diff: Diffing boehm_gc/libatomic_ops-1.2/src/atomic_ops/sysdeps
cvs diff: Diffing boehm_gc/libatomic_ops-1.2/src/atomic_ops/sysdeps/gcc
cvs diff: Diffing boehm_gc/libatomic_ops-1.2/src/atomic_ops/sysdeps/hpc
cvs diff: Diffing boehm_gc/libatomic_ops-1.2/src/atomic_ops/sysdeps/ibmc
cvs diff: Diffing boehm_gc/libatomic_ops-1.2/src/atomic_ops/sysdeps/icc
cvs diff: Diffing boehm_gc/libatomic_ops-1.2/src/atomic_ops/sysdeps/msftc
cvs diff: Diffing boehm_gc/libatomic_ops-1.2/src/atomic_ops/sysdeps/sunc
cvs diff: Diffing boehm_gc/libatomic_ops-1.2/tests
cvs diff: Diffing boehm_gc/tests
cvs diff: Diffing boehm_gc/windows-untested
cvs diff: Diffing boehm_gc/windows-untested/vc60
cvs diff: Diffing boehm_gc/windows-untested/vc70
cvs diff: Diffing boehm_gc/windows-untested/vc71
cvs diff: Diffing browser
cvs diff: Diffing bytecode
cvs diff: Diffing compiler
Index: compiler/Mmakefile
===================================================================
RCS file: /home/mercury/mercury1/repository/mercury/compiler/Mmakefile,v
retrieving revision 1.96
diff -u -b -r1.96 Mmakefile
--- compiler/Mmakefile	24 Oct 2007 09:21:14 -0000	1.96
+++ compiler/Mmakefile	22 Dec 2007 18:28:11 -0000
@@ -13,7 +13,8 @@
 # Override the settings in ../Mmake.workspace so that in debugging grades we
 # do not include mer_mdbcomp.init twice in the list of files passed to mkinit.
 #
-C2INITFLAGS = --trace-init-file $(BROWSER_DIR)/$(BROWSER_LIB_NAME).init \
+C2INITFLAGS = \
+	--trace-init-file $(BROWSER_DIR)/$(BROWSER_LIB_NAME).init \
 	--trace-init-file $(SSDB_DIR)/$(SSDB_LIB_NAME).init
 
 -include Mmake.compiler.params
@@ -34,7 +35,11 @@
 
 MERCURY_MAIN_MODULES = top_level mlds_to_gcc
 
-VPATH = $(LIBRARY_DIR) $(MDBCOMP_DIR) $(BROWSER_DIR) $(SSDB_DIR) \
+VPATH = \
+	$(LIBRARY_DIR) \
+	$(MDBCOMP_DIR) \
+	$(BROWSER_DIR) \
+	$(SSDB_DIR) \
 	$(ANALYSIS_DIR)
 
 #-----------------------------------------------------------------------------#
Index: compiler/layout_out.m
===================================================================
RCS file: /home/mercury/mercury1/repository/mercury/compiler/layout_out.m,v
retrieving revision 1.93
diff -u -b -r1.93 layout_out.m
--- compiler/layout_out.m	12 Sep 2007 06:21:07 -0000	1.93
+++ compiler/layout_out.m	22 Dec 2007 15:41:12 -0000
@@ -2126,9 +2126,9 @@
     DeepExcpVars = deep_excp_slots(TopCSDSlot, MiddleCSDSlot,
         OldOutermostSlot),
     io.write_int(TopCSDSlot, !IO),
-    io.write_string(",\n\t", !IO),
+    io.write_string(",\n", !IO),
     io.write_int(MiddleCSDSlot, !IO),
-    io.write_string(",\n\t", !IO),
+    io.write_string(",\n", !IO),
     io.write_int(OldOutermostSlot, !IO),
     io.write_string("\n};\n", !IO),
     decl_set_insert(decl_data_addr(layout_addr(LayoutName)), !DeclSet).
cvs diff: Diffing compiler/notes
cvs diff: Diffing debian
cvs diff: Diffing debian/patches
cvs diff: Diffing deep_profiler
cvs diff: Diffing deep_profiler/notes
cvs diff: Diffing doc
cvs diff: Diffing extras
cvs diff: Diffing extras/base64
cvs diff: Diffing extras/cgi
cvs diff: Diffing extras/complex_numbers
cvs diff: Diffing extras/complex_numbers/samples
cvs diff: Diffing extras/complex_numbers/tests
cvs diff: Diffing extras/concurrency
cvs diff: Diffing extras/curs
cvs diff: Diffing extras/curs/samples
cvs diff: Diffing extras/curses
cvs diff: Diffing extras/curses/sample
cvs diff: Diffing extras/dynamic_linking
cvs diff: Diffing extras/error
cvs diff: Diffing extras/fixed
cvs diff: Diffing extras/gator
cvs diff: Diffing extras/gator/generations
cvs diff: Diffing extras/gator/generations/1
cvs diff: Diffing extras/graphics
cvs diff: Diffing extras/graphics/easyx
cvs diff: Diffing extras/graphics/easyx/samples
cvs diff: Diffing extras/graphics/mercury_allegro
cvs diff: Diffing extras/graphics/mercury_allegro/examples
cvs diff: Diffing extras/graphics/mercury_allegro/samples
cvs diff: Diffing extras/graphics/mercury_allegro/samples/demo
cvs diff: Diffing extras/graphics/mercury_allegro/samples/mandel
cvs diff: Diffing extras/graphics/mercury_allegro/samples/pendulum2
cvs diff: Diffing extras/graphics/mercury_allegro/samples/speed
cvs diff: Diffing extras/graphics/mercury_glut
cvs diff: Diffing extras/graphics/mercury_opengl
cvs diff: Diffing extras/graphics/mercury_tcltk
cvs diff: Diffing extras/graphics/samples
cvs diff: Diffing extras/graphics/samples/calc
cvs diff: Diffing extras/graphics/samples/gears
cvs diff: Diffing extras/graphics/samples/maze
cvs diff: Diffing extras/graphics/samples/pent
cvs diff: Diffing extras/lazy_evaluation
cvs diff: Diffing extras/lex
cvs diff: Diffing extras/lex/samples
cvs diff: Diffing extras/lex/tests
cvs diff: Diffing extras/log4m
cvs diff: Diffing extras/logged_output
cvs diff: Diffing extras/moose
cvs diff: Diffing extras/moose/samples
cvs diff: Diffing extras/moose/tests
cvs diff: Diffing extras/mopenssl
cvs diff: Diffing extras/morphine
cvs diff: Diffing extras/morphine/non-regression-tests
cvs diff: Diffing extras/morphine/scripts
cvs diff: Diffing extras/morphine/source
cvs diff: Diffing extras/net
cvs diff: Diffing extras/odbc
cvs diff: Diffing extras/posix
cvs diff: Diffing extras/posix/samples
cvs diff: Diffing extras/quickcheck
cvs diff: Diffing extras/quickcheck/tutes
cvs diff: Diffing extras/references
cvs diff: Diffing extras/references/samples
cvs diff: Diffing extras/references/tests
cvs diff: Diffing extras/solver_types
cvs diff: Diffing extras/solver_types/library
cvs diff: Diffing extras/trailed_update
cvs diff: Diffing extras/trailed_update/samples
cvs diff: Diffing extras/trailed_update/tests
cvs diff: Diffing extras/windows_installer_generator
cvs diff: Diffing extras/windows_installer_generator/sample
cvs diff: Diffing extras/windows_installer_generator/sample/images
cvs diff: Diffing extras/xml
cvs diff: Diffing extras/xml/samples
cvs diff: Diffing extras/xml_stylesheets
cvs diff: Diffing java
cvs diff: Diffing java/runtime
cvs diff: Diffing library
cvs diff: Diffing mdbcomp
cvs diff: Diffing profiler
cvs diff: Diffing robdd
cvs diff: Diffing runtime
Index: runtime/mercury_deep_profiling.c
===================================================================
RCS file: /home/mercury/mercury1/repository/mercury/runtime/mercury_deep_profiling.c,v
retrieving revision 1.28
diff -u -b -r1.28 mercury_deep_profiling.c
--- runtime/mercury_deep_profiling.c	15 Nov 2007 03:57:59 -0000	1.28
+++ runtime/mercury_deep_profiling.c	22 Dec 2007 15:40:13 -0000
@@ -1695,6 +1695,8 @@
         MR_fatal_error("MR_write_out_proc_layout_from_void: no proc_id\n");
     }
 
+    fprintf(stderr, "UNWRITTEN proc layout %p:\n", pl);
+
     procid = &proc_layout->MR_sle_proc_id;
     if (MR_PROC_ID_IS_UCI(*procid)) {
         fprintf(stderr, "uci %s/%s/%s/%s/%d/%d\n",
@@ -1713,8 +1715,6 @@
             procid->MR_proc_user.MR_user_arity,
             procid->MR_proc_user.MR_user_mode);
     }
-
-    fprintf(stderr, "UNWRITTEN proc layout %p\n", pl);
 }
 
 static void
cvs diff: Diffing runtime/GETOPT
cvs diff: Diffing runtime/machdeps
cvs diff: Diffing samples
cvs diff: Diffing samples/c_interface
cvs diff: Diffing samples/c_interface/c_calls_mercury
cvs diff: Diffing samples/c_interface/cplusplus_calls_mercury
cvs diff: Diffing samples/c_interface/mercury_calls_c
cvs diff: Diffing samples/c_interface/mercury_calls_cplusplus
cvs diff: Diffing samples/c_interface/mercury_calls_fortran
cvs diff: Diffing samples/c_interface/simpler_c_calls_mercury
cvs diff: Diffing samples/c_interface/simpler_cplusplus_calls_mercury
cvs diff: Diffing samples/c_interface/standalone_c
cvs diff: Diffing samples/diff
cvs diff: Diffing samples/muz
cvs diff: Diffing samples/rot13
cvs diff: Diffing samples/solutions
cvs diff: Diffing samples/solver_types
cvs diff: Diffing samples/tests
cvs diff: Diffing samples/tests/c_interface
cvs diff: Diffing samples/tests/c_interface/c_calls_mercury
cvs diff: Diffing samples/tests/c_interface/cplusplus_calls_mercury
cvs diff: Diffing samples/tests/c_interface/mercury_calls_c
cvs diff: Diffing samples/tests/c_interface/mercury_calls_cplusplus
cvs diff: Diffing samples/tests/c_interface/mercury_calls_fortran
cvs diff: Diffing samples/tests/c_interface/simpler_c_calls_mercury
cvs diff: Diffing samples/tests/c_interface/simpler_cplusplus_calls_mercury
cvs diff: Diffing samples/tests/diff
cvs diff: Diffing samples/tests/muz
cvs diff: Diffing samples/tests/rot13
cvs diff: Diffing samples/tests/solutions
cvs diff: Diffing samples/tests/toplevel
cvs diff: Diffing scripts
Index: scripts/c2init.in
===================================================================
RCS file: /home/mercury/mercury1/repository/mercury/scripts/c2init.in,v
retrieving revision 1.50
diff -u -b -r1.50 c2init.in
--- scripts/c2init.in	24 Oct 2007 09:21:17 -0000	1.50
+++ scripts/c2init.in	23 Dec 2007 01:28:02 -0000
@@ -90,34 +90,42 @@
 		$mercury_stdlib_dir/modules/$GRADE/$SSDB_LIB_NAME.init"
 fi
 MERCURY_TRACE_LIB_MODS="$MERCURY_TRACE_LIB_MODS $trace_init_files"
+MERCURY_SSDB_LIB_MODS="$MERCURY_SSDB_LIB_MODS $ssdb_init_files"
 
+MERCURY_ALL_LIB_MODS="$MERCURY_MOD_LIB_MODS"
+
+# XXX why does this test $trace_opt instead of $trace?
 case "$trace_opt" in
 	-t)
 		init_opt="-i"
-		MERCURY_ALL_LIB_MODS0="$MERCURY_MOD_LIB_MODS\
+		MERCURY_ALL_LIB_MODS="$MERCURY_ALL_LIB_MODS\
 			$MERCURY_TRACE_LIB_MODS"
 		;;
-	*)
-		MERCURY_ALL_LIB_MODS0="$MERCURY_MOD_LIB_MODS"
-		;;
 esac
 
 case $ss_debug in
 	true)
-		MERCURY_ALL_LIB_MODS="$MERCURY_ALL_LIB_MODS0\
-			$MERCURY_SSDB_LIB_MODS"
+		ssdb=true
 		;;
-	false)
-		MERCURY_ALL_LIB_MODS="$MERCURY_ALL_LIB_MODS0"
+esac
+
+case $ssdb in
+	true)
+		MERCURY_ALL_LIB_MODS="$MERCURY_ALL_LIB_MODS\
+			$MERCURY_SSDB_LIB_MODS"
 		;;
 esac
 
+# XXX: $EXTRA_INIT_FILES is not set or used anywhere by our code, and users
+# can't set it since it is not documented anywhere, so its value should
+# always be the empty string.
+
 case $# in
 	0) exec $MKINIT -c"$maxcalls" $init_opt $trace_opt \
 		$library_opt $defentry_opt $extra_inits_opt \
 		-g "$GRADE" -o "$init_c_file" $experimental_complexity_opt \
 		$extra_init_dirs $always_exec_init_opts \
-		$EXTRA_INIT_FILES $TRACE_INIT_FILES $MERCURY_ALL_LIB_MODS
+		$EXTRA_INIT_FILES $MERCURY_ALL_LIB_MODS
 	   ;;
 	*) exec $MKINIT -c"$maxcalls" $init_opt $trace_opt \
 		$library_opt $defentry_opt $extra_inits_opt \
@@ -125,6 +133,6 @@
 		-r "$runtime_flags" \
 		$extra_init_dirs $always_exec_init_opts \
 		"$@" \
-		$EXTRA_INIT_FILES $TRACE_INIT_FILES $MERCURY_ALL_LIB_MODS
+		$EXTRA_INIT_FILES $MERCURY_ALL_LIB_MODS
 	   ;;
 esac
Index: scripts/parse_ml_options.sh-subr.in
===================================================================
RCS file: /home/mercury/mercury1/repository/mercury/scripts/parse_ml_options.sh-subr.in,v
retrieving revision 1.12
diff -u -b -r1.12 parse_ml_options.sh-subr.in
--- scripts/parse_ml_options.sh-subr.in	24 Feb 2006 07:11:21 -0000	1.12
+++ scripts/parse_ml_options.sh-subr.in	23 Dec 2007 01:42:35 -0000
@@ -25,6 +25,7 @@
 verbose=false
 allow_undef=false
 trace=false
+ssdb=false
 readline=true
 case $FULLARCH in
 	*-win95|*-winnt|*-win32|*-cygwin32|*-cygwin)
@@ -57,11 +58,13 @@
 defentry_opt=""
 init_opt=""
 trace_opt=""
+ssdb_opt=""
 library_opt=""
 extra_inits_opt=""
 always_exec_init_opts=""
 extra_init_dirs=""
 trace_init_files=""
+ssdb_init_files=""
 init_c_file="-"
 runtime_flags=""
 experimental_complexity=""
@@ -259,6 +262,14 @@
 		trace=false
 		trace_opt=""
 		;;
+	--ssdb)
+		ssdb=true
+		ssdb_opt="--ssdb"
+		;;
+	--no-ssdb)
+		ssdb=false
+		ssdb_opt=""
+		;;
 	-r|--readline)
 		readline=true
 		;;
@@ -438,6 +449,9 @@
 	--trace-init-file)
 		trace_init_files="$trace_init_files $2"; shift;;
 
+	--ssdb-init-file)
+		ssdb_init_files="$ssdb_init_files $2"; shift;;
+
 	--runtime-flags)
 		runtime_flags="$runtime_flags $2"; shift;;
 
cvs diff: Diffing slice
cvs diff: Diffing ssdb
cvs diff: Diffing tests
cvs diff: Diffing tests/benchmarks
cvs diff: Diffing tests/debugger
cvs diff: Diffing tests/debugger/declarative
cvs diff: Diffing tests/dppd
cvs diff: Diffing tests/general
cvs diff: Diffing tests/general/accumulator
cvs diff: Diffing tests/general/string_format
cvs diff: Diffing tests/general/structure_reuse
cvs diff: Diffing tests/grade_subdirs
cvs diff: Diffing tests/hard_coded
cvs diff: Diffing tests/hard_coded/exceptions
cvs diff: Diffing tests/hard_coded/purity
cvs diff: Diffing tests/hard_coded/sub-modules
cvs diff: Diffing tests/hard_coded/typeclasses
cvs diff: Diffing tests/invalid
cvs diff: Diffing tests/invalid/purity
cvs diff: Diffing tests/misc_tests
cvs diff: Diffing tests/mmc_make
cvs diff: Diffing tests/mmc_make/lib
cvs diff: Diffing tests/par_conj
cvs diff: Diffing tests/recompilation
cvs diff: Diffing tests/tabling
cvs diff: Diffing tests/term
cvs diff: Diffing tests/trailing
cvs diff: Diffing tests/valid
cvs diff: Diffing tests/warnings
cvs diff: Diffing tools
Index: tools/bootcheck
===================================================================
RCS file: /home/mercury/mercury1/repository/mercury/tools/bootcheck,v
retrieving revision 1.202
diff -u -b -r1.202 bootcheck
--- tools/bootcheck	22 Nov 2007 05:33:53 -0000	1.202
+++ tools/bootcheck	22 Dec 2007 15:32:41 -0000
@@ -151,6 +151,13 @@
         Keep the Deep.data files resulting from the bootcheck. The
         usual behavior is to delete them in the interest of conserving
         disk space.
+    --deep-all-write
+        In deep profiling grades, write out the profiling tree on every
+        invocation. (The default is to write it out only for every 25th
+        invocation, on average.
+    --deep-debug
+        Turn on the flag that enables debugging output from the deep
+        profiling runtime.
     --progress
         Create a timestamp file when finishing the execution of each test case,
         to allow the user to monitor the bootcheck's progress. The timestamp
@@ -208,6 +215,8 @@
 coverage_test=false
 disable_debug_libs=false
 delete_deep_data=true
+deep_write_sample=true
+deep_debug=false
 progress=false
 
 if test -f .KEEP_OBJS
@@ -387,6 +396,12 @@
     --keep-deep-data)
         delete_deep_data=false ;;
 
+    --deep-all-write)
+        deep_write_sample=false ;;
+
+    --deep-debug)
+        deep_debug=true ;;
+
     --progress)
         progress=true ;;
 
@@ -555,14 +570,22 @@
 #
 # When the profiling data is written out, however, we also want to test
 # writing out the program representation.
-MERCURY_OPTIONS="$MERCURY_OPTIONS --deep-random-write=25 --deep-procrep-file"
+if $deep_write_sample
+then
+    MERCURY_OPTIONS="$MERCURY_OPTIONS --deep-random-write=25"
+fi
+
+if $deep_debug
+then
+    MERCURY_OPTIONS="$MERCURY_OPTIONS --deep-debug-file"
+fi
+
+MERCURY_OPTIONS="$MERCURY_OPTIONS --deep-procrep-file"
 
 if $trace_count
 then
     MERCURY_OPTIONS="$MERCURY_OPTIONS --trace-count-if-exec=mercury_compile"
     export MERCURY_OPTIONS
-else
-    true
 fi
 
 if $coverage_test
@@ -574,8 +597,6 @@
     export MERCURY_OPTIONS
     # Check whether we can compile the slice directory.
     copy_slice=true
-else
-    true
 fi
 
 NEW_MERCURY_OPTIONS="$MERCURY_OPTIONS --mdb-disable-progress"
cvs diff: Diffing trace
cvs diff: Diffing util
cvs diff: Diffing vim
cvs diff: Diffing vim/after
cvs diff: Diffing vim/ftplugin
cvs diff: Diffing vim/syntax
--------------------------------------------------------------------------
mercury-reviews mailing list
Post messages to:       mercury-reviews at csse.unimelb.edu.au
Administrative Queries: owner-mercury-reviews at csse.unimelb.edu.au
Subscriptions:          mercury-reviews-request at csse.unimelb.edu.au
--------------------------------------------------------------------------
Previous message: [m-rev.] for post-commit review: clean up ssdb.m
Next message: [m-rev.] for post-commit review: fix deep profiling bootstrap bug
Messages sorted by: [ date ] [ thread ] [ subject ] [ author ]
More information about the reviews mailing list