[m-rev.] for review: resets and statistics for tabling

[Julien Fischer]

On Thu, 8 Jun 2006, Zoltan Somogyi wrote:

> Provide a mechanism for collecting statistics about tabling operations,
> and provide a much more convenient mechanism for resetting tables.
>
> Since it would too complex to do this while preserving the capability
> of setting --tabling-via-extra-args to no, eliminate that capability
> and the option. That option was useful only for measurements of the
> performance boost from setting --tabling-via-extra-args to yes in any case,
> so users lose functionality.
>

I think you mean that they lose *no* functionality there.

> Previously, the only way to debug the low level details of the tabling
> mechanism was to build a runtime with a specific C macro (MR_TABLE_DEBUG)
> and link with that runtime; this was cumbersome. Change that so that
> every one of the debuggable tabling macro has a bool argument that says

s/macro/macros/

> whether debugging is enabled or not. The compiler can then set this to
> MR_TRUE if the new option --table-debug is given, and to MR_FALSE otherwise.
> If set to MR_FALSE, the C compiler should optimize away the debug code,
> with zero impact on program size or speed.

So MR_TABLE_DEBUG is now gone?

> Since these changes to macros require nontrivial bootstrapping, which we don't
> want to do unnecessarily, modify the interface of the tabling macros as
> required to support size limits on tables. This diff also implements the
> parsing of size limit specifications on tables, but does not implement them
> yet; that is for a future change.
>
> To make the syntax simpler, this diff deletes the free-standing
> fast_loose_memo pragma. The same functionality is now available with a
> fast_loose annotation on an ardinary memo pragma.
>

s/ardinary/ordinary/

> Make a bunch of changes to improve readability and maintainability
> in the process. These mostly take the form of renaming ambiguous and/or
> not sufficiently expressive function symbols.
>
> runtime/mercury_stack_layout.h:
> runtime/mercury_tabling.h:
> 	Move the description of structure of tables from mercury_stack_layout.h
> 	to mercury_tabling.h, since we now need it for statistics even if
> 	execution tracing is not enabled.
>
> 	Modify those data structures to have room for the statistics.
>
> 	Don't distinguish "strict", "fast_loose" and "specified" memoing
> 	as separate eval methods; treat them as just different kinds
> 	of the same eval method: "memo".
>
> 	Remove underscores from the names of some types that the style guide
> 	says shouldn't be there.
>
> runtime/mercury_tabling_preds.h:
> runtime/mercury_tabling_macros.h:
> 	Modify the approach we use to macros that implement the predicates

s/to/for/

> 	of library/table_builtin.m. Instead of selecting between debug and
> 	nondebug based on whether MR_TABLE_DEBUG is defined or not, add
> 	an explicit argument controlling this to each debuggable macro.
> 	The advantage of the new arrangement is that it scales. Another
> 	argument controls whether we are computing statistics (and if yes,
> 	where do we put it), and a third argument controls whether we maintain
> 	back links in the tries and hash tables (this last argument is present
> 	but is ignored for now).
>
> 	Since the values of the arguments will be known when the .c files
> 	containing calls to these macros are compiled, we pay the space and
> 	time cost of debugging, statistics gathering and the maintenance of
> 	back links if and only we need the revelant functionality.
>
> 	Provide macros for limited backward compatibility with the old set
> 	of macros; these allow workspaces created by old compilers to work
> 	with the new macros in the runtime. The old macros followed the
> 	naming scheme MR_table_*, the new ones are named MR_tbl_*.
>
> runtime/mercury_table_int_fix_index_body.h:
> runtime/mercury_table_int_start_index_body.h:
> runtime/mercury_table_type_body.h:
> 	New files containing parts of the old mercury_tabling.c. Each of these
> 	files contains the body of the functions that used to be in
> 	mercury_tabling.c. The new mercury_tabling.c #includes each of these
> 	files more than once, to provide more than one variant of the old
> 	function. These variants differ in aspects such as whether debugging
> 	is enabled or statistics is being collected. Each variant therefore
> 	incurs only the time costs it needs to. (We pay the space cost of
> 	having all these variants all the time of course, but this cost
> 	is negligible.)
>
> runtime/mercury_tabling_stats_defs.h:
> runtime/mercury_tabling_stats_nodefs.h:
> runtime/mercury_tabling_stats_undefs.h:
> 	New files that serve as wrappers around the newly #included files,
> 	controlling how they handle statistics.
>
> runtime/mercury_tabling.c:
> 	Delete functions now in the new files, and #include them instead.
> 	Delete the data structures that used to contain summary statistics;
> 	the new approach keeps statistics in compiler-generated,
> 	procedure-specific data structures.
>
> runtime/mercury_trace_base.c:
> 	Use the new versions of the tabling macros to access the I/O table.
>
> runtime/mercury_type_info.h:
> 	Update some documentation for the movement of code out of
> 	mercury_tabling.c.
>
> runtime/mercury_types.h:
> 	Provide forward declarations of the identifiers denoting the new types
> 	in mercury_tabling.h.
>
> runtime/mercury_grade.h:
> 	Increment the exec trace version number, since we have changed
> 	a part of the exec trace structure.
>
> runtime/mercury_bootstrap.h:
>

This part of the log message appears to be missing.

> runtime/mercury_hash_lookup_or_add_body.h:
> 	Fix comment.
>
> runtime/Mmakefile:
> 	Mention the new files and the dependencies that involve them.
>
> library/table_builtin.m:
> 	Provide a type for representing statistics and a predicate for
> 	printing statistics.
>
> 	Use the updated versions of the macros in
> 	runtime/mercury_tabling_preds.h.
>
> compiler/prog_item.m:
> 	Change representation of tabling pragmas to allow room for the new
> 	attributes.
>
> 	Allow an item to be marked as being generated by the compiler
> 	as a result of a pragma memo attribute. We use this for the reset
> 	and statistics predicates.
>
> compiler/mercury_to_mercury.m:
> 	Write out the new attributes of the tabling pragma.
>
> compiler/prog_data.m:
> compiler/hlds_data.m:
> 	Change the cons_id that used to refer to a procedure's call table root
> 	to refer to the entirety of the new data structure now containing it.
> 	The compiler now needs a way to refer to the other components of this
> 	new data structure, since it contains the statistics.
>
> 	As in the runtime, don't distinguish "strict", "fast_loose" and
> 	"specified" memoing as separate eval methods; treat them as just
> 	different kinds of the same eval method: "memo".
>
> 	Rename some of the uses of the function symbols "c", "java", "il".
>
> compiler/hlds_pred.m:
> 	Add an extra field in proc_infos for storing any tabling attributes.
>
> 	Change the existing proc_info field that records information about
> 	the kinds of arguments of tabled procedures to record the information
> 	needed by the debugger too. This was needed to allow us to shift all
> 	the RTTI for procedure-specific tables (as opposed to the RTTI for
> 	the global I/O table) from mercury_stack_layout.h to mercury_tabling.h
> 	without duplicating the data (which would be a maintenance problem).
>
> 	Reformat some comments to make them easier to read.
>
> compiler/layout.m:
> compiler/layout_out.m:
> 	Delete the part of the exec trace information that used to record
> 	RTTI for tables, since this information is not generated only as
> 	part of the debugger data structures anymore.
>
> compiler/prog_io_pragma.m:
> 	Recognize the updated syntax for tabling pragmas.
>
> compiler/add_pragma.m:
> 	When processing tabling pragmas for inclusion in the HLDS, create
> 	any reset and statistics predicates they ask for.
>
> compiler/make_hlds_passes.m:
> 	Export a predicate now needed by add_pragma.m.
>
> 	Handle the new attributes on tabling pragmas
>
> compiler/globals.m:
> 	Change the function symbols of the types describing backends and
> 	foreign languages to say what they are. Previously, both types (as well
> 	as several others) included the function symbol "c"; now, they are
> 	target_c and lang_c respectively.
>
> compiler/table_gen.m:
> 	Implement the changes described at the top.
>
> 	When passing around varsets and vartypes, pass the arguments in the
> 	standard order.
>
> compiler/goal_util.m:
> compiler/hlds_goal.m:
> 	When passing around varsets and vartypes, pass the arguments in the
> 	standard order.
>
> compiler/rtti.m:
> 	Provide types for representing the runtime's data structures for
> 	tabling (which are now significantly more complex than a single word)
> 	and predicates for manipulating them, for use by both the ml and ll
> 	backends.
>
> compiler/llds.m:
> 	Replace the comp_gen_c_var type with the tabling_info_struct type,
> 	which contains the information needed to create the per-procedure
> 	tabling data structures.
>
> 	Replace references to call tables with references to the various
> 	components of the new tabling data structures.
>
> compiler/llds_out.m:
> 	Add code to write out tabling_info_structs.
>
> 	Delete the code required for old, hacky way of resetting tables.


... for *the* old ...

Given the size of this change I would be happy for you to commit it now and
deal with any review comments post-commit provided that the compiler boostraps
in both a lowlevel grade and in a hlc grade - (given that the are also
substantial runtime changes you should also check that the runtime still
compiles in various grades, e.g. .profdeep, .mm, hlc.agc etc.)

I'll review the rest of this sometime in the next few days.

Julien.
--------------------------------------------------------------------------
mercury-reviews mailing list
post:  mercury-reviews at cs.mu.oz.au
administrative address: owner-mercury-reviews at cs.mu.oz.au
unsubscribe: Address: mercury-reviews-request at cs.mu.oz.au Message: unsubscribe
subscribe:   Address: mercury-reviews-request at cs.mu.oz.au Message: subscribe
--------------------------------------------------------------------------