[m-rev.] for review: make the deep profiler work again

[Zoltan Somogyi]

On 18-Nov-2002, Fergus Henderson <fjh at cs.mu.OZ.AU> wrote:
> Please at least add an XXX comment.

I added a comment.

> There should be a comment explaining why you close all those streams.

Done.

> Have you tested that?  I don't think that will work, since io.m will
> call MR_fatal_error (or throw an exception) for any attempts to close
> the pre-defined standard I/O streams.

It worked despite the fatal errors. The server cleaned up after itself,
and the next query attempted to start the server again :-(

Fergus I discussed this, and decided that the best fix was to change io.m
to allow the standard I/O streams to be closed. The diff for that fix,
and a diff to the NEWS file announcing the deep profiler follow.

> > +#ifdef	MP_DEBUG_MDPROF_SIGNAL
> 
> That configuration macro should be documented in runtime/mercury_conf_param.h.

It already was, though due to a typo it was MR_DEBUG_MDPROF_SIGNAL, not
MP_DEBUG_MDPROF_SIGNAL. Fixed.

> > +++ tests/general/Mmakefile	16 Nov 2002 03:20:33 -0000
> > +MCFLAGS += --no-optimize-duplicate-calls
> 
> I don't think that is a good idea.  Maybe you included this part accidentally?

Yes; I backed it out.

I will commit this change on the main branch after a last bootcheck,
and on the relase branch after it has been tested on the main branch.

Zoltan.

Index: NEWS
===================================================================
RCS file: /home/mercury1/repository/mercury/NEWS,v
retrieving revision 1.279
diff -u -b -r1.279 NEWS
--- NEWS	4 Nov 2002 02:14:24 -0000	1.279
+++ NEWS	20 Nov 2002 05:37:06 -0000
@@ -32,6 +32,12 @@
 * Declarative debugging.
 * Support for transparent retries across I/O.
 
+A new profiler, which we call the Mercury deep profiler or mdprof:
+* Supports both time and memory profiling.
+* Gathers information about individual call sites as well as procedures.
+* Eliminates the assumption that all calls to a procedure have equal cost.
+* Allows users to explore the gathered data interactive with a web browser.
+
 Numerous minor improvements to the Mercury standard library.
 
 A new testing tool in the extras distribution.
@@ -426,6 +432,47 @@
   io__open_file, io__write_string etc) idempotent. This means that a given
   call to e.g. io__open_file will open the specified file only once,
   even if retry commands cause the call to be executed more than once.
+
+A new profiler, which we call the Mercury deep profiler or mdprof:
+
+* The old Mercury profiler is based on the technology of the standard Unix
+  profiler gprof. This technology makes the assumption that all calls to a
+  given C function (in Mercury, a given function or predicate in a given mode)
+  have the same cost, whether the cost being measured is CPU time, memory cells
+  allocated, memory words allocated etc. In C programs, this assumption is
+  usually close enough to correct for the output of gprof to be useful. In
+  Mercury, due to the presence of parametric polymorphism, different call sites
+  are far more likely to have distinct performance characteristics than in C,
+  so the output of a gprof-style profiler is usually not accurate enough to be
+  useful.
+
+  The new profiler records, for each of its measurements, not just the current
+  predicate/function and its caller, but the entire chain of ancestors. This
+  "deep context" is what gives the profiler its name. Actually, to keep
+  overheads down, we don't walk the stack at every measurement; we just
+  associate the current context with each measurement, and update the current
+  context when it changes. Given this fact, it costs very little extra to
+  record measurements on every aspect of performance (counts of calls, exits,
+  fails and redos, counts of memory cells and memory words allocated, and time
+  spent). We thus have only one deep profiling grade component, .profdeep,
+  whereas the old profiler has several, each which gathers a given subset
+  of these measurements.
+
+* The deep context recorded by the deep profiler records the identities of
+  the call sites as well as the identities of predicates and functions
+  in the list of ancestors. If a predicate p contains two calls to predicate q,
+  this allows the deep profiler to report that one call to q costs next to
+  nothing while the other one is a major performance problem.
+
+* The deep profiler gathers so much data that giving it to the user all at once
+  would swamp the user with too much information. We therefore implemented the
+  deep profiler as a CGI program. Users can use thus use a web browser to
+  explore the information contained in profiling data files.
+
+* Further information about the deep profiler is available in the paper
+  "Deep profiling: engineering a profiler for a declarative programming
+  language" by Thomas C. Conway and Zoltan Somogyi, available from our web
+  site.
 
 Changes to the compiler back-ends:
 
Index: library/io.m
===================================================================
RCS file: /home/mercury1/repository/mercury/library/io.m,v
retrieving revision 1.276
diff -u -b -r1.276 io.m
--- library/io.m	6 Nov 2002 04:30:47 -0000	1.276
+++ library/io.m	20 Nov 2002 04:21:23 -0000
@@ -4146,15 +4146,6 @@
 void
 mercury_close(MercuryFile* mf)
 {
-	if (mf == &mercury_stdin ||
-	    mf == &mercury_stdout ||
-	    mf == &mercury_stderr ||
-	    mf == &mercury_stdin_binary ||
-	    mf == &mercury_stdout_binary)
-	{
-		mercury_io_error(mf,
-			""attempt to close stdin, stdout or stderr"");	
-	} else {
 		if (MR_CLOSE(*mf) < 0) {
 			mercury_io_error(mf, ""error closing file: %s"",
 				strerror(errno));
@@ -4206,6 +4197,16 @@
 #endif /* ! MR_NEW_MERCURYFILE_STRUCT */
 
 #ifndef MR_CONSERVATIVE_GC
+	if (mf == &mercury_stdin ||
+	    mf == &mercury_stdout ||
+	    mf == &mercury_stderr ||
+	    mf == &mercury_stdin_binary ||
+	    mf == &mercury_stdout_binary)
+	{
+		/*
+		** The memory for these streams is allocated statically,
+		** so there is nothing to free.
+	} else {
   		/*
 		** For the accurate GC or no GC cases,
 		** we need to explicitly deallocate the memory here,
@@ -4215,9 +4216,8 @@
 		** as a foreign_type.
 		*/
   		MR_GC_free(mf);
-#endif /* !MR_CONSERVATIVE_GC */
-
 	}
+#endif /* !MR_CONSERVATIVE_GC */
 }
 
 ").
@@ -4227,18 +4227,8 @@
 static void
 mercury_close(MR_MercuryFile mf)
 {
-        if (mf == mercury_stdin ||
-            mf == mercury_stdout ||
-            mf == mercury_stderr ||
-            mf == mercury_stdin_binary ||
-            mf == mercury_stdout_binary)
-        {
-                // XXX We should throw an exception here.
-                ;	
-        } else {
                 mf->stream->Close();
                 mf->stream = NULL;
-        }
 }
 
 ").
--------------------------------------------------------------------------
mercury-reviews mailing list
post:  mercury-reviews at cs.mu.oz.au
administrative address: owner-mercury-reviews at cs.mu.oz.au
unsubscribe: Address: mercury-reviews-request at cs.mu.oz.au Message: unsubscribe
subscribe:   Address: mercury-reviews-request at cs.mu.oz.au Message: subscribe
--------------------------------------------------------------------------