[m-dev.] ROTD build

[Fergus Henderson]

On 08-Dec-2003, Fergus Henderson <fjh at cs.mu.OZ.AU> wrote:
> On 04-Dec-2003, Peter Ross <pro at missioncriticalit.com> wrote:
> > The ROTD build appears to be exhausting available memory on jupiter,
> > when attempting to build options.m.
> 
> Also on lots of our other nightly test systems.
> On some (e.g. roy) it is exceeding CPU time rather than memory.
> 
> > Can someone increase the memory available to the autobuilder?
> 
> I don't think that is a good idea.  We probably have a serious
> performance regression and we should investigate which change
> caused the problem and see if/how it can be avoided.

Looking at the log files, the change which introduced the problem has
been committed some time between 8:35pm December 1st Melbourne time
(test passed on earth.cs.mu.oz.au) and 1:49am December 3rd (test failed
on roy.cs.mu.oz.au).

Enclosed below is a list of all of the possible changes which could have
caused the problem.  But I think it is almost certainly due to the first
one, Simon's abstract equivalence types change, since the other changes
should only affect the MLDS-based back-ends, not the LLDS back-end.

Simon, could you please investigate what is causing the performance
problem, and commit a fix or work-around ASAP?  If the new equiv_type_hlds
pass is triggering the problem, then a possible work-around might be to
temporarily turn that pass off (or make it optional).

Here's the change which I think caused the problem:

 | **************************************
 | Date:	Tuesday December 2, 2003 @ 2:55
 | Author:	stayl
 | 
 | Estimated hours taken: 100
 | Branches: main
 | 
 | Make definitions of abstract types available when generating
 | code for importing modules.  This is necessary for the .NET
 | back-end, and for `:- pragma export' on the C back-end.
 | 
 | compiler/prog_data.m:
 | compiler/modules.m:
 | compiler/make.dependencies.m:
 | compiler/recompilation.version.m:
 | 	Handle implementation sections in interface files.
 | 
 | 	There is a new pseudo-declaration `abstract_imported'
 | 	which is applied to items from the implementation
 | 	section of an interface file.  `abstract_imported'
 | 	items may not be used in the error checking passes
 | 	for the curent module.
 | 
 | compiler/equiv_type_hlds.m:
 | compiler/notes/compiler_design.html:
 | 	New file.
 | 
 | 	Go over the HLDS expanding all types fully after
 | 	semantic checking has been run.
 | 
 | compiler/mercury_compile.m:
 | 	Add the new pass.
 | 
 | 	Don't write the `.opt' file if there are any errors.	
 | 
 | compiler/instmap.m:
 | 	Add a predicate instmap_delta_map_foldl to apply
 | 	a procedure to all insts in an instmap.
 | 
 | compiler/equiv_type.m:
 | 	Export predicates for use by equiv_type_hlds.m
 | 
 | 	Reorder arguments so state variables and higher-order
 | 	programming can be used.
 | 
 | compiler/prog_data.m:
 | compiler/prog_io_pragma.m:
 | compiler/make_hlds.m:
 | compiler/mercury_to_mercury.m:
 | 	Handle `:- pragma foreign_type' as a form of type
 | 	declaration rather than a pragma.
 | 
 | compiler/hlds_data.m:
 | compiler/*.m:
 | 	Add a field to the type_info_cell_constructor cons_id
 | 	to identify the type_ctor, which is needed by
 | 	equiv_type_hlds.m.
 | 
 | compiler/module_qual.m:
 | 	Donn't allow items from the implementation section of
 | 	interface files to match items in the current module.
 | 
 | compiler/*.m:
 | tests/*/*.m:
 | 	Add missing imports which only became apparent with
 | 	the bug fixes above.
 | 
 | 	Remove unnecessary imports which only became apparent with
 | 	the bug fixes above.
 | 
 | tests/hard_coded/Mmakefile:
 | tests/hard_coded/export_test2.{m,exp}:
 | 	Test case.
 | 
 | tests/invalid/Mmakefile:
 | tests/invalid/missing_interface_import2.{m,err_exp}:
 | 	Test case.

For completeness, here are the ones that I think can't have
caused the problem:

 | **************************************
 | Date:	Tuesday December 2, 2003 @ 8:50
 | Author:	petdr
 | 
 | Estimated hours taken: 0.25
 | Branches: main
 | 
 | library/io.m:
 | 	Fix an XXX about implemting io__get_stream_id directly as a
 | 	function.  It dates back from when we couldn't generate
 | 	functions in MC++, which I fixed over a year ago.
 | 
 | **************************************
 | Date:	Tuesday December 2, 2003 @ 9:31
 | Author:	petdr
 | 
 | Estimated hours taken: 8
 | Branches: main
 | 
 | The comparison and unification function pointers stored in the
 | type_ctor_info must be pointers to functions where all the arguments
 | are boxed.  This wasn't occuring on the IL backend for types which
 | are value types.
 | 
 | compiler/rtti_to_mlds.m:
 | 	Call ml_gen_closure_wrapper to construct a version of the
 | 	comparison and unification functions where the arguments are
 | 	boxed.  The wrapper function simply unboxes the arguments and
 | 	calls the actual special pred.
 | 
 | compiler/ml_closure_gen.m:
 | 	Adapt ml_gen_closure_wrapper so that it can generate a wrapper
 | 	function for special preds.
 | 
 | tests/hard_coded/Mmakefile:
 | tests/hard_coded/equality_pred_which_requires_boxing.exp:
 | tests/hard_coded/equality_pred_which_requires_boxing.m:
 | 	Add a test case for this bug.
 | 
 | **************************************
 | Date:	Tuesday December 2, 2003 @ 15:41
 | Author:	fjh
 | 
 | Estimated hours taken: 0.1
 | Branches: main
 | 
 | tools/submit_patch:
 | 	Back out the previous change, which was clearly nonsensical --
 | 	inserting IL code in a shell script!?! -- and must have been
 | 	committed accidentally (by petdr).
 | 
 | File: submit_patch     	Status: Up-to-date
 |    Working revision:	1.7	Tue Dec  2 04:40:21 2003
 |    lines: +0 -24   (383)
 | 
 | **************************************
 | Date:	Tuesday December 2, 2003 @ 15:41
 | Author:	fjh
 | 
 | Estimated hours taken: 0.5
 | Branches: main
 | 
 | tools/test_mercury:
 | 	Use /home/mercury/public/installed_w3 rather than
 | 	/home/mercury5/installed_w3.
 | 	Report an error if we can't copy the ROTD distribution
 | 	to the ftp or web sites.
 | 
 | **************************************
 | Date:	Tuesday December 2, 2003 @ 15:47
 | Author:	fjh
 | 
 | Estimated hours taken: 0.1
 | Branches: main
 | 
 | tutorial/types-n-things.m4:
 | 	Fix syntax error (extraneous comma) reported by
 | 	"Jesse A. Tov" <tov at post.harvard.edu>.
 | 
 | **************************************
 | Date:	Tuesday December 2, 2003 @ 19:55
 | Author:	petdr
 | 
 | Estimated hours taken: 0.1
 | Branches: main
 | 
 | library/rtti_implementation.m:
 | 	Swap the order of some C# statements in typeinfo_is_variable/2
 | 	so that the line which can cause an exception occurs first.
 | 
 | **************************************
 | Date:	Tuesday December 2, 2003 @ 20:16
 | Author:	fjh
 | 
 | Estimated hours taken: 0.5.
 | Branches: main
 | 
 | compiler/accumulator.m:
 | 	Add an XXX comment about this module not handling exceptions
 | 	and non-termination correctly.
 | 
 | **************************************
 | Date:	Tuesday December 2, 2003 @ 21:02
 | Author:	fjh
 | 
 | Estimated hours taken: 6
 | Branches: main
 | 
 | More work on the Java back-end.  The standard library now compiles
 | in grade `java', and hello world (the version using io__write_string,
 | and linked against the standard library in library/*.m, not the
 | hand-coded one in java/library/*.java) now works!
 | 
 | compiler/make_hlds.m:
 | 	Ignore `pragma type_spec' declarations for the Java back-end.
 | 	This works around a problem where javac was failing to compile
 | 	some of our generated code due to it overflowing limits on
 | 	file name length for the names of the .class files for some
 | 	nested classes.
 | 
 | compiler/mlds_to_java.m:
 | 	Add some comments.  Add myself to the "main authors" list.
 | 
 | library/string.m:
 | 	Provide Java definitions of string__first_char and
 | 	string__unsafe_index.  (These are needed for string.append,
 | 	which is used by private_builtin.sorry.)
 | 
 | library/io.m:
 | 	Provide Java definitions of io__write_{string,int,char,float}/3.
 | 
 | java/runtime/TypeCtorInfo_Struct.java:
 | 	Fix a cut-and-paste error.
 | 
 | java/runtime/TypeInfo_Struct.java:
 | 	Improve the implementation of the TypeInfo_Struct(Object)
 | 	constructor so that it doesn't throw exceptions during
 | 	the initialization of the standard library.
 | 
 | java/runtime/FA_TypeInfo_Struct1.java:
 | 	Make this type inherit from TypeInfo_Struct.

-- 
Fergus Henderson <fjh at cs.mu.oz.au>  |  "I have always known that the pursuit
The University of Melbourne         |  of excellence is a lethal habit"
WWW: <http://www.cs.mu.oz.au/~fjh>  |     -- the last words of T. S. Garp.
--------------------------------------------------------------------------
mercury-developers mailing list
Post messages to:       mercury-developers at cs.mu.oz.au
Administrative Queries: owner-mercury-developers at cs.mu.oz.au
Subscriptions:          mercury-developers-request at cs.mu.oz.au
--------------------------------------------------------------------------