[m-rev.] for review: deep profiling on the web page
Zoltan Somogyi
zs at cs.mu.OZ.AU
Wed Aug 1 17:21:32 AEST 2001
For review by anyone familiar with our web site's design. In particular, I'd
like to know (1) whether my use of mdprof\??echo $root? does what I want it
to do (i.e. result in the string mdprof?/home/mercury5/installed_w3 in the
HTML generated by PHP), and (2) what is the best place in the non-news part
of the web site to link to the new deep profiling page from.
Zoltan.
Announce the deep profiler paper and its demo.
news/newsdb.inc:
Add an entry announcing the paper and the demo.
information/include/papers.inc:
Add the paper to the list of papers.
Fix some capitalization inconsistencies.
information/papers/mu_01_24.ps.gz:
The deep profiling paper.
information/deep_demo.php3:
A new page, giving a broad outline of the objectives of the deep
profiler and a link to the demos.
information/Makefile:
Mention the new page.
information/deep_demo/*:
Data files for the demo.
cvs diff: Diffing .
cvs diff: Diffing bin
cvs diff: Diffing contact
cvs diff: Diffing contact/include
cvs diff: Diffing download
cvs diff: Diffing download/include
cvs diff: Diffing download/patches
cvs diff: Diffing htdig
cvs diff: Diffing htdig/include
cvs diff: Diffing images
cvs diff: Diffing include
cvs diff: Diffing information
Index: information/Makefile
===================================================================
RCS file: /home/mercury1/repository/w3/information/Makefile,v
retrieving revision 1.14
diff -u -b -r1.14 Makefile
--- information/Makefile 2001/03/12 19:48:36 1.14
+++ information/Makefile 2001/08/01 07:16:09
@@ -3,6 +3,7 @@
ROOT=..
HTML= benchmarks.html \
comparison_with_haskell.html \
+ deep_demo.html \
developer.html \
documentation.html \
events.html \
Index: information/deep_demo.php3
===================================================================
RCS file: deep_demo.php3
diff -N deep_demo.php3
--- /dev/null Fri Dec 1 02:25:58 2000
+++ deep_demo.php3 Thu Jul 19 16:07:46 2001
@@ -0,0 +1,10 @@
+<HTML>
+<?
+ $menu="Information";
+ $title="Deep profiler demo";
+ $dir="information";
+ $root="..";
+ $include="deep_demo.inc";
+ include "$root/include/template.inc"
+?>
+</HTML>
cvs diff: Diffing information/bench
cvs diff: Diffing information/deep_demo
Index: information/deep_demo/eliza.data
===================================================================
RCS file: eliza.data
diff -N eliza.data
Binary files /dev/null and eliza.data differ
Index: information/deep_demo/eliza.data.contour
===================================================================
RCS file: eliza.data.contour
diff -N eliza.data.contour
--- /dev/null Fri Dec 1 02:25:58 2000
+++ eliza.data.contour Wed Aug 1 17:03:54 2001
@@ -0,0 +1,59 @@
+all array
+all assoc_list
+all bag
+all benchmarking
+all bimap
+all bintree
+all bintree_set
+all bitmap
+all bool
+all bt_array
+all builtin
+all char
+all counter
+all dir
+all enum
+all eqvclass
+all exception
+all float
+all gc
+all getopt
+all graph
+all group
+all hash_table
+all int
+all integer
+all io
+all lexer
+all library
+all list
+all map
+all math
+all multi_map
+all ops
+all parser
+all pprint
+all pqueue
+all private_builtin
+all prolog
+all queue
+all random
+all rational
+all rbtree
+all relation
+all require
+all set
+all set_bbbtree
+all set_ordlist
+all set_unordlist
+all sparse_bitset
+all stack
+all std_util
+all store
+all string
+all table_builtin
+all term
+all term_io
+all time
+all tree234
+all varset
Index: information/deep_demo/make_hlds.data
===================================================================
RCS file: make_hlds.data
diff -N make_hlds.data
Binary files /dev/null and make_hlds.data differ
Index: information/deep_demo/make_hlds.data.contour
===================================================================
RCS file: make_hlds.data.contour
diff -N make_hlds.data.contour
--- /dev/null Fri Dec 1 02:25:58 2000
+++ make_hlds.data.contour Wed Aug 1 17:03:47 2001
@@ -0,0 +1,59 @@
+all array
+all assoc_list
+all bag
+all benchmarking
+all bimap
+all bintree
+all bintree_set
+all bitmap
+all bool
+all bt_array
+all builtin
+all char
+all counter
+all dir
+all enum
+all eqvclass
+all exception
+all float
+all gc
+all getopt
+all graph
+all group
+all hash_table
+all int
+all integer
+all io
+all lexer
+all library
+all list
+all map
+all math
+all multi_map
+all ops
+all parser
+all pprint
+all pqueue
+all private_builtin
+all prolog
+all queue
+all random
+all rational
+all rbtree
+all relation
+all require
+all set
+all set_bbbtree
+all set_ordlist
+all set_unordlist
+all sparse_bitset
+all stack
+all std_util
+all store
+all string
+all table_builtin
+all term
+all term_io
+all time
+all tree234
+all varset
Index: information/deep_demo/monte.data
===================================================================
RCS file: monte.data
diff -N monte.data
Binary files /dev/null and monte.data differ
cvs diff: Diffing information/developers
cvs diff: Diffing information/dotnet
cvs diff: Diffing information/events
cvs diff: Diffing information/events/images
cvs diff: Diffing information/include
Index: information/include/deep_demo.inc
===================================================================
RCS file: deep_demo.inc
diff -N deep_demo.inc
--- /dev/null Fri Dec 1 02:25:58 2000
+++ deep_demo.inc Wed Aug 1 16:51:09 2001
@@ -0,0 +1,123 @@
+<h2>The Mercury deep profiler</h2>
+
+<h3>Objective</h3>
+
+The old Mercury profiler, mprof, was a straightforward clone
+of the standard Unix profiling tool, gprof.
+However, while gprof is quite useful in profiling C programs,
+we found the usefulness of mprof in profiling Mercury programs
+to be severely limited.
+The main reason is that
+gprof and mprof assume that all calls to a function or predicate
+have roughly the same cost.
+This is usually close enough to the truth in C programs
+for the output of gprof to be useful.
+In Mercury, it is usually very far from truth,
+because Mercury programs make much greater use of polymorphism
+than C programs do.
+
+For example, the most expensive predicates in the Mercury compiler are
+the predicates for searching 2-3-4 trees and for inserting into 2-3-4 trees.
+However, these predicates are called (indirectly)
+from many hundreds of places in the Mercury compiler,
+some of which handle bigger trees than others.
+
+We designed the deep profiler specifically to meet the needs
+of programs written in programming languages such as Mercury,
+programs whose characteristics include
+
+<ul>
+<li>
+frequent use of recursion, including mutual recursion;
+<li>
+frequent use of polymorphism (in Mercury's case, parametric polymorphism),
+including nested polymorphism
+(where one builds polymorphic functions/predicates
+from polymorphic building blocks)
+and polymorphism within callers
+(where a function/predicate calls another function/predicate
+from two or more call sites
+that have different purposes and different performance characteristics);
+<li>
+frequent use of higher order constructs
+(calls through higher order variables, and method calls);
+<li>
+nested use of higher order constructs
+(where a function/predicate can make a higher order call
+that directly or indirectly results in a call to itself);
+<li>
+use of a bidirectional foreign-language interface.
+</ul>
+
+The key to our solution of these problems is to
+associate with each profiling measurement a very detailed context:
+essentially a representation of the entire chain
+of the ancestor functions or predicates and their call sites
+at the time of the measurement,
+but compressing sequences of (mutually) recursive ancestor calls.
+The main challenge was keeping the required overhead within bounds.
+Deep profiling slows down programs by a factor between two and three,
+with a factor of 2.4 being reasonably typical.
+Since this is only slightly higher than the overheads of profiling techniques
+that yield significantly less detailed and less accurate data,
+we believe we succeeded.
+
+<h3>Architecture</h3>
+
+Like most profilers, the Mercury deep profiler works
+by instrumenting the program to be profiled,
+having the instrumented program record its profiling data in a file,
+and postprocessing the contents of this file.
+
+You can ask for a program to be compiled with deep profiling instrumentation
+by compiling it in a grade that includes the grade component "profdeep",
+e.g. asm_fast.gc.profdeep.
+Deep profiling is not compatible with the old Mercury profiling grades,
+and it is not (yet) implemented for grades (such as hlc grades)
+that use the compiler's MLDS back end.
+
+Programs compiled with deep profiling can generate large amounts of data,
+since deep profiling yields very detailed information.
+We have therefore implemented the postprocessing program as a web server.
+Its input is a stream of requests,
+with each request specifying a data file,
+the part the user wants to view,
+and his or her preferences about the format of the display.
+The server generates, for each request,
+a web page containing the requested information,
+which will typically contain links that generate other requests.
+
+<h3>Demo</h3>
+
+We have made three profiling data files available for you to explore.
+Each of the links below will take you to pages
+that are dynamically created by the deep profiling tool
+from one of these data files.
+
+<p>
+
+<ul>
+<li>
+<a href = "http://venus.cs.mu.oz.au/cgi-bin/mdprof\??echo $root?/information/deep_demo/make_hlds.data">mmc -O2 make_hlds.m</a>:
+the profiled executable is the Mercury compiler,
+and during the profiling run, it task was to compile make_hlds.m,
+the largest module (about 8300 lines) in its own source code.
+Though large, this module is straightforward to compile.
+<li>
+<a href = "http://venus.cs.mu.oz.au/cgi-bin/mdprof\??echo $root?/information/deep_demo/eliza.data">mmc -O2 eliza.m</a>:
+the profiled executable is the Mercury compiler,
+and during the profiling run, it task was to compile eliza.m,
+the Mercury implementation of the classic "AI" program,
+which is available in the samples directory in the Mercury distribution.
+Though small (625 lines), this module contains some constructs
+that stress the Mercury compiler.
+<li>
+<a href = "http://venus.cs.mu.oz.au/cgi-bin/mdprof\??echo $root?/information/deep_demo/monte.data">monte</a>:
+the profiled executable is a program that
+calculates the volume of a 3D shape using Monte Carlo methods.
+This program uses higher order constructs quite intensively.
+</ul>
+
+<h3>Availability</h3>
+
+The Mercury deep profiler is available in the Mercury releases of the day.
Index: information/include/papers.inc
===================================================================
RCS file: /home/mercury1/repository/w3/information/include/papers.inc,v
retrieving revision 1.22
diff -u -b -r1.22 papers.inc
--- information/include/papers.inc 2001/05/18 11:15:41 1.22
+++ information/include/papers.inc 2001/07/19 05:59:02
@@ -16,11 +16,53 @@
<ul>
<li>
<strong>
-Practical Aspects for a working Compile Time Garbage Collection System
+Deep profiling:
+engineering a profiler for a declarative programming language
+</strong>
+<br>
+Thomas C. Conway and Zoltan Somogyi
+<br>
+<em>
+Technical Report 2001/24,
+</em>
+Department of Computer Science, University of Melbourne,
+Melbourne, Australia, July 2001, 61 pages.
+<a href = "papers/mu_01_24.ps.gz">
+Available here (207K).
+</a>
+<p>
+Declarative programs differ from imperative programs in several respects,
+the main ones being their heavy use of recursion,
+of various forms of polymorphism, and of higher order.
+Existing profilers tend not to produce very useful information
+in the presence of these constructs.
+We present a new profiling technique we call \emph{deep profiling}
+that yields detailed and useful information about programs
+even in the presence of these constructs,
+information that is significantly richer
+than the output of other profilers.
+<p>
+The heart of deep profiling is a source-to-source transformation.
+We have implemented this transformation and its associated infrastructure
+in the compiler for Mercury, a purely declarative logic programming language.
+While our measurements of this implementation
+show that deep profiling has slightly greater overhead
+than some other profiling techniques,
+the wealth of information it provides
+makes this extra overhead worthwhile.
+The deep profiling algorithms themselves
+are applicable to most other language styles,
+including imperative, object-oriented, and functional languages.
+<p>
+
+<ul>
+<li>
+<strong>
+Practical aspects for a working compile time garbage collection system
for Mercury
</strong>
<br>
-Nancy Mazur, Peter Ross, Gerda Janssens, Maurice Bruynooghe
+Nancy Mazur, Peter Ross, Gerda Janssens and Maurice Bruynooghe
<br>
<em>
Technical Report CW310
@@ -111,7 +153,7 @@
<li>
<strong>
-Making Mercury Programs Tail Recursive
+Making Mercury programs tail recursive
</strong>
<br>
Peter Ross, David Overton and Zoltan Somogyi
cvs diff: Diffing information/papers
Index: information/papers/mu_01_24.ps.gz
===================================================================
RCS file: mu_01_24.ps.gz
diff -N mu_01_24.ps.gz
Binary files /dev/null and mu_01_24.ps.gz differ
cvs diff: Diffing information/reports
cvs diff: Diffing mailing-lists
cvs diff: Diffing mailing-lists/include
cvs diff: Diffing mailing-lists/mercury-ads
cvs diff: Diffing mailing-lists/mercury-ads/include
cvs diff: Diffing mailing-lists/mercury-developers
cvs diff: Diffing mailing-lists/mercury-developers/include
cvs diff: Diffing mailing-lists/mercury-reviews
cvs diff: Diffing mailing-lists/mercury-reviews/include
cvs diff: Diffing mailing-lists/mercury-users
cvs diff: Diffing mailing-lists/mercury-users/include
cvs diff: Diffing news
Index: news/newsdb.inc
===================================================================
RCS file: /home/mercury1/repository/w3/news/newsdb.inc,v
retrieving revision 1.57
diff -u -b -r1.57 newsdb.inc
--- news/newsdb.inc 2001/05/18 11:15:43 1.57
+++ news/newsdb.inc 2001/08/01 07:21:00
@@ -22,6 +22,19 @@
$newsdb = array(
+"1 August 2001" => array("New paper and demo",
+
+"We have a new paper available
+from our <A HREF=\"information/papers.html\">papers page</A>,
+which describes the design and implementation of the Mercury deep profiler.
+This new profiler generates profiling information
+that is significantly more accurate and more detailed
+than the information produced by most other profilers.
+A <A HREF=\"information/deep_demo.html\">
+description and demo of the deep profiler</A>
+is also available."
+),
+
"18 May 2001" => array("New paper",
"We have a new paper available
--------------------------------------------------------------------------
mercury-reviews mailing list
post: mercury-reviews at cs.mu.oz.au
administrative address: owner-mercury-reviews at cs.mu.oz.au
unsubscribe: Address: mercury-reviews-request at cs.mu.oz.au Message: unsubscribe
subscribe: Address: mercury-reviews-request at cs.mu.oz.au Message: subscribe
--------------------------------------------------------------------------
More information about the reviews
mailing list