[m-rev.] For review: Make profiler feedback analysis easier to use.

Paul Bone pbone at csse.unimelb.edu.au
Mon Nov 14 17:39:19 AEDT 2011


Various changes to make feedback analysis (for automatic parallelism) easier to
use.

These changes mean that a normal installation with a deep profiling and a
low-level C parallel grade can be used for automatic parallelism.

compiler/options.m:
    Rename --profile-for-implicit-parallelism to --profile-for-feedback.
    --profile-for-implicit-parallelism is now retained as a synonym.

    Enable --coverage-profiling by default, it is only checked if
    --deep-profiling is enabled.

    Make corrections to the commented-out documentation for the developer-only
    coverage profiling options.

compiler/handle_options.m:
    Conform to changes in options.m

    Remove the set of options implied by --coverage-profiling since they're
    already set correctly in options.m.

doc/user_guide.texi:
    Added extra concept index declarations to entries for automatic parallelism
    and feedback analysis.

    Conform to changes in options.m

library/Mmakefile:
    Compile the standard library with --profile-optimized

runtime/mercury_deep_profiling.c:
    Always write out the Procrep.data file when writing out the Deep.data file.

runtime/mercury_wrapper.[ch]:
    Remove the --deep-procrep-file MERCURY_OPTIONS option, it is now always
    enabled.  (In the future the procrep data may be included inside the
    Deep.data file).

tools/bootcheck:
    Remove the --deep-procrep-file option from the bootcheck script.

Index: compiler/handle_options.m
===================================================================
RCS file: /home/mercury1/repository/mercury/compiler/handle_options.m,v
retrieving revision 1.371
diff -u -p -b -r1.371 handle_options.m
--- compiler/handle_options.m	17 Oct 2011 04:31:28 -0000	1.371
+++ compiler/handle_options.m	14 Nov 2011 06:33:45 -0000
@@ -1348,34 +1348,9 @@ convert_options_to_globals(OptionTable0,
 
     option_implies(target_debug, strip, bool(no), !Globals),
 
-    % Profile for implicit parallelism implies a particular coverage
-    % profiling configuration, since that's what is supported.
-    globals.lookup_bool_option(!.Globals, profile_for_implicit_parallelism,
-        ProfForImplicitParallelism),
-    (
-        ProfForImplicitParallelism = yes,
-        globals.set_option(coverage_profiling, bool(yes), !Globals),
-        globals.set_option(coverage_profiling_static, bool(no), !Globals),
-        globals.set_option(profile_deep_coverage_after_goal, bool(yes),
+    % Profile for feedback requires coverage profiling.
+    option_implies(profile_for_feedback, coverage_profiling, bool(yes),
             !Globals),
-        globals.set_option(profile_deep_coverage_branch_ite, bool(yes),
-            !Globals),
-        globals.set_option(profile_deep_coverage_branch_switch, bool(yes),
-            !Globals),
-        globals.set_option(profile_deep_coverage_branch_disj, bool(yes),
-            !Globals),
-
-        % Disabling these two options causes the coverage profiling
-        % transformation to make a single backwards traversal.  This makes
-        % coverage information propagation easier at the expense of
-        % inserting more coverage points.
-        globals.set_option(profile_deep_coverage_use_portcounts, bool(no),
-            !Globals),
-        globals.set_option(profile_deep_coverage_use_trivial, bool(no),
-            !Globals)
-    ;
-        ProfForImplicitParallelism = no
-    ),
 
     % At the moment coverage profiling is not compatible with the tail
     % recursion preservation optimization used by the deep profiler.
@@ -1389,14 +1364,13 @@ convert_options_to_globals(OptionTable0,
     % user can re-enable it with the `--profile-optimized' option.  Leave
     % inlineing enabled when profiling for implicit parallelism.
     %
+    option_implies(profile_for_feedback, prof_optimized, bool(yes), !Globals),
     globals.lookup_bool_option(!.Globals, prof_optimized, ProfOptimized),
     (
         ProfOptimized = no,
-        ProfForImplicitParallelism = no
-    ->
         option_implies(profile_deep, allow_inlining, bool(no), !Globals)
     ;
-        true
+        ProfOptimized = yes
     ),
 
     % Perform a simplification pass before the profiling passes if one of
Index: compiler/options.m
===================================================================
RCS file: /home/mercury1/repository/mercury/compiler/options.m,v
retrieving revision 1.708
diff -u -p -b -r1.708 options.m
--- compiler/options.m	8 Nov 2011 02:22:42 -0000	1.708
+++ compiler/options.m	14 Nov 2011 06:33:45 -0000
@@ -360,9 +360,9 @@
     ;       profile_deep_coverage_use_portcounts
     ;       profile_deep_coverage_use_trivial
 
-            % Turn on flags relevant for profiler directed implicit
-            % parallelism.
-    ;       profile_for_implicit_parallelism
+            % Turn on flags relevant for profiler directed feedback analysis.
+            % Currently the only feedback analysis is automatic parallelism.
+    ;       profile_for_feedback
 
     ;       use_zeroing_for_ho_cycles
     ;       use_lots_of_ho_specialization
@@ -1291,7 +1291,7 @@ option_defaults_2(compilation_model_opti
     use_activation_counts               -   bool(no),
     pre_prof_transforms_simplify        -   bool(no),
     pre_implicit_parallelism_simplify   -   bool(no),
-    coverage_profiling                  -   bool(no),
+    coverage_profiling                  -   bool(yes),
     coverage_profiling_via_calls        -   bool(no),
     coverage_profiling_static           -   bool(no),
     profile_deep_coverage_after_goal    -   bool(yes),
@@ -1300,7 +1300,7 @@ option_defaults_2(compilation_model_opti
     profile_deep_coverage_branch_disj   -   bool(yes),
     profile_deep_coverage_use_portcounts -  bool(no),
     profile_deep_coverage_use_trivial   -   bool(no),
-    profile_for_implicit_parallelism    -   bool(no),
+    profile_for_feedback                -   bool(no),
     use_zeroing_for_ho_cycles           -   bool(yes),
     use_lots_of_ho_specialization       -   bool(no),
     deep_profile_tail_recursion         -   bool(no),
@@ -2207,7 +2207,9 @@ long_option("profile-deep-coverage-use-p
 long_option("profile-deep-coverage-use-trivial",
                     profile_deep_coverage_use_trivial).
 long_option("profile-for-implicit-parallelism",
-                    profile_for_implicit_parallelism).
+                    profile_for_feedback).
+long_option("profile-for-feedback",
+                    profile_for_feedback).
 long_option("use-zeroing-for-ho-cycles",
                     use_zeroing_for_ho_cycles).
 long_option("use-lots-of-ho-specialization",
@@ -4318,34 +4320,30 @@ options_help_compilation_model -->
 %       "\tSimilar to `--memory-profiling', except that it only",
 %       "\tgathers memory usage information, not call counts.",
 
-        "--coverage-profiling",
-        "\tEnable coverage profiling, implies --deep-profiling (above).",
+        "--no-coverage-profiling",
+        "\tDisable coverage profiling.",
 % The following options are for implementors only (intended for experiments).
 %       "--coverage-profiling-via-calls",
 %       "\tUse calls to implement coverage points, not inline foreign code.",
 
-%       "--no-coverage-profiling-dynamic",
+%       "--coverage-profiling-static",
 %       "\tDisable dynamic coverage profiling, this uses less memory and may ",
 %       "\tbe faster.",
 
 %       "Switches to effect coverage profiling (part of deep profiling). ",
 %       "they enable different types of coverage points.",
 
-%       "--profile-deep-coverage-may-fail",
-%       "\tEnable coverage points after goals that may fail.",
-%       "--profile-deep-coverage-multi",
-%       "\tEnable coverage points after goals that may succeed more than ",
-%       "\tonce.",
-%       "--profile-deep-coverage-any",
-%       "\tEnable coverage points after goals that may succeed more than ",
-%       "\tonce or fail.",
-%       "--profile-deep-coverage-branch-if",
-%       "\tEnable coverage points at the beginning of if branches.",
-%       "--profile-deep-coverage-branch-switch",
-%       "\tEnable coverage points at the beginning of switch branches.",
-%       "--profile-deep-coverage-branch-disj",
-%       "\tEnable coverage points at the beginning of disjunction branches.",
+%       "--no-profile-deep-coverage-after-goal",
+%       "\tDisable coverage points after goals.",
+%       "--no-profile-deep-coverage-branch-ite",
+%       "\tDisable coverage points at the beginning of then and else",
+%       "\tbranches.",
+%       "--no-profile-deep-coverage-branch-switch",
+%       "\tDisable coverage points at the beginning of switch branches.",
+%       "--no-profile-deep-coverage-branch-disj",
+%       "\tDisable coverage points at the beginning of disjunction branches.",
 
+%       I beleive these options are broken - pbone.
 %       "Switches to tune the coverage profiling pass, useful for ",
 %       "debugging.",
 %
@@ -4355,9 +4353,11 @@ options_help_compilation_model -->
 %       "--no-profile-deep-coverage-use-trivial",
 %       "\tTurn off usage of trivial goal information",
 
-        "--profile-for-implicit-parallelism",
+        "--profile-for-feedback",
         "\tSelect deep profiling options suitable for profiler directed",
-        "\timplicit parallelism",
+        "\timplicit parallelism.",
+        "\t--profile-for-implicit-parallelism is a deprecated synonym for",
+        "\tthis option",
 
         "--record-term-sizes-as-words\t\t(grade modifier: `.tsw')",
         "\tAugment each heap cells with its size in words.",
Index: doc/user_guide.texi
===================================================================
RCS file: /home/mercury1/repository/mercury/doc/user_guide.texi,v
retrieving revision 1.642
diff -u -p -b -r1.642 user_guide.texi
--- doc/user_guide.texi	8 Nov 2011 23:19:49 -0000	1.642
+++ doc/user_guide.texi	14 Nov 2011 06:33:45 -0000
@@ -6344,6 +6344,8 @@ if it finds any syntax errors in the pro
 % @sp 1
 % @item --halt-at-auto-parallel-failure
 % @findex --halt-at-auto-parallel-failure
+% @cindex Automatic parallelism
+% @cindex Profiler feedback
 % This option causes the compiler to halt if it cannot perform
 % an auto-parallelization requested by a feedback file.
 
@@ -7927,21 +7929,26 @@ Enable deep profiling by inserting the a
 This option is only supported by the low-level C back-end.
 
 @sp 1
- at item @code{--coverage-profiling}
- at findex --coverage-profiling
+ at item @code{--no-coverage-profiling}
+ at findex --no-coverage-profiling
 @cindex Coverage Profiling
-Enable coverage profiling by inserting coverage points, these are used to
-measure how execution moves through a procedure.  This option only effects
-programs built  in deep profiling grades.
+Disable coverage profiling.
+Coverage profiling is part of the deep profiler and only used in deep profiling
+grades,
+it inserts coverage points that are used to measure how execution moves through
+a procedure.
 
 @sp 1
- at item @code{--profile-for-implicit-parallelism}
+ at item @code{--profile-for-feedback}
 @findex --profile-for-implicit-parallelism
- at cindex Coverage Profiling
- at cindex Implicit Parallelism
+ at findex --profile-for-feedback
+ at cindex Coverage profiling
+ at cindex Profiler feedback
+ at cindex Automatic parallelism
 Enable options in the deep profiler that generate suitable profiles for use
-with profile directed implicit parallelism.  This option only effects programs
-built in deep profiling grades.
+with profile directed feedback analysis.
+Currently the only feedback analysis is used for automatic parallelism.
+This option only effects programs built in deep profiling grades.
 
 @ignore
     Coverage profiling is experimental, some options below tune coverage
@@ -7953,34 +7960,24 @@ Switches to effect coverage profiling (p
 different types of coverage points.
 
 @sp 1
- at item @code{--profile-deep-coverage-may-fail}
- at findex --profile-deep-coverage-may-fail
-Enable coverage points after goals that may fail.
-
- at sp 1
- at item @code{--profile-deep-coverage-multi}
- at findex --profile-deep-coverage-multi
-Enable coverage points after goals that may succeed more than once.
-
- at sp 1
- at item @code{--profile-deep-coverage-any}
- at findex --profile-deep-coverage-any
-Enable coverage points after goals that may succeed more than once or fail.
+ at item @code{--no-profile-deep-coverage-after-goals}
+ at findex --no-profile-deep-coverage-after-goals
+Disable insertion of coverage points after goals where coverage is unknown.
 
 @sp 1
- at item @code{--profile-deep-coverage-branch-if}
- at findex --profile-deep-coverage-branch-if
-Enable coverage points at the beginning of if branches.
+ at item @code{--no-profile-deep-coverage-branch-ite}
+ at findex --no-profile-deep-coverage-branch-if
+Disable coverage points at the beginning of then and else branches.
 
 @sp 1
- at item @code{--profile-deep-coverage-branch-switch}
- at findex --profile-deep-coverage-branch-switch
-Enable coverage points at the beginning of switch branches.
+ at item @code{--no-profile-deep-coverage-branch-switch}
+ at findex --no-profile-deep-coverage-branch-switch
+Disable coverage points at the beginning of switch branches.
 
 @sp 1
- at item @code{--profile-deep-coverage-branch-disj}
- at findex --profile-deep-coverage-branch-disj
-Enable coverage points at the beginning of disjunction branches.
+ at item @code{--no-profile-deep-coverage-branch-disj}
+ at findex --no-profile-deep-coverage-branch-disj
+Disable coverage points at the beginning of disjunction branches.
 
 @sp 1
 Switches to tune the coverage profiling pass, useful for
@@ -9532,6 +9529,8 @@ using the specified distance value.
 @sp 1
 @item --implicit-parallelism
 @findex --implicit-parallelism
+ at cindex Automatic parallelism
+ at cindex Profiler feedback
 Introduce parallel conjunctions where it could be worthwhile (implicit
 parallelism) using deep profiling feedback information generated by
 mdprof_create_feedback. The profiling feedback file can be specified using the
@@ -9540,8 +9539,10 @@ mdprof_create_feedback. The profiling fe
 @sp 1
 @item --feedback-file
 @findex --feedback-file
+ at cindex Automatic parallelism
+ at cindex Profiler feedback
 Use the specified profiling feedback file
-which may currently only be processed for implicit parallelism.
+which may currently only be processed for automatic parallelism.
 
 @c @sp 1
 @c @item --loop-control
Index: library/Mmakefile
===================================================================
RCS file: /home/mercury1/repository/mercury/library/Mmakefile,v
retrieving revision 1.179
diff -u -p -b -r1.179 Mmakefile
--- library/Mmakefile	10 May 2011 00:28:15 -0000	1.179
+++ library/Mmakefile	14 Nov 2011 06:33:45 -0000
@@ -167,6 +167,13 @@ endif
 
 #-----------------------------------------------------------------------------#
 
+# The standard library is compiled with --profile-optimized so that it is
+# compatible with applications compiled with --profile-for-feedback.
+
+MCFLAGS += --profile-optimized
+
+#-----------------------------------------------------------------------------#
+
 # targets
 
 .PHONY: all
Index: runtime/mercury_deep_profiling.c
===================================================================
RCS file: /home/mercury1/repository/mercury/runtime/mercury_deep_profiling.c,v
retrieving revision 1.38
diff -u -p -b -r1.38 mercury_deep_profiling.c
--- runtime/mercury_deep_profiling.c	12 Nov 2011 11:42:03 -0000	1.38
+++ runtime/mercury_deep_profiling.c	14 Nov 2011 06:33:45 -0000
@@ -397,14 +397,11 @@ MR_write_out_profiling_tree(void)
             MR_MDPROF_DATA_FILENAME, strerror(errno));
     }
 
-    procrep_fp = NULL;
-    if (MR_output_deep_procrep_file) {
         procrep_fp = fopen(MR_MDPROF_PROCREP_FILENAME, "wb+");
         if (procrep_fp == NULL) {
             MR_fatal_error("cannot open `%s' for writing: %s",
                 MR_MDPROF_PROCREP_FILENAME, strerror(errno));
         }
-    }
 
 #ifdef  MR_DEEP_PROFILING_DEBUG
     if (MR_deep_prof_debug_file_flag) {
@@ -431,9 +428,7 @@ MR_write_out_profiling_tree(void)
     MR_write_fixed_size_int(deep_fp, 0);
     MR_write_fixed_size_int(deep_fp, 0);
 
-    if (procrep_fp != NULL) {
         MR_write_out_procrep_id_string(procrep_fp);
-    }
 
 #ifdef  MR_CLOCK_TICKS_PER_SECOND
     ticks_per_sec = MR_CLOCK_TICKS_PER_SECOND;
@@ -492,13 +487,11 @@ MR_write_out_profiling_tree(void)
         MR_deep_data_output_error("cannot close", MR_MDPROF_DATA_FILENAME);
     }
 
-    if (procrep_fp != NULL) {
         putc(MR_no_more_modules, procrep_fp);
         if (fclose(procrep_fp) != 0) {
             MR_deep_data_output_error("cannot close",
                 MR_MDPROF_PROCREP_FILENAME);
         }
-    }
 
 #ifdef MR_DEEP_PROFILING_STATISTICS
     if (! MR_print_deep_profiling_statistics) {
@@ -805,7 +798,6 @@ void
 MR_write_out_module_proc_reps_start(FILE *procrep_fp,
     const MR_ModuleCommonLayout *module_common)
 {
-    if (procrep_fp != NULL) {
         int         i;
 
         putc(MR_next_module, procrep_fp);
@@ -814,15 +806,12 @@ MR_write_out_module_proc_reps_start(FILE
         for (i = 0; i < module_common->MR_mlc_string_table_size; i++) {
             putc(module_common->MR_mlc_string_table[i], procrep_fp);
         }
-    }
 }
 
 void
 MR_write_out_module_proc_reps_end(FILE *procrep_fp)
 {
-    if (procrep_fp != NULL) {
         putc(MR_no_more_procs, procrep_fp);
-    }
 }
 
 void
@@ -1013,7 +1002,6 @@ MR_write_out_proc_static(FILE *deep_fp, 
         MR_write_out_call_site_static(deep_fp, &ps->MR_ps_call_sites[i]);
     }
 
-    if (procrep_fp != NULL) {
         const MR_uint_least8_t  *bytecode;
 
         /*
@@ -1046,7 +1034,6 @@ MR_write_out_proc_static(FILE *deep_fp, 
                 putc(bytecode[bytenum], procrep_fp);
             }
         }
-    }
 }
 
 void
Index: runtime/mercury_wrapper.c
===================================================================
RCS file: /home/mercury1/repository/mercury/runtime/mercury_wrapper.c,v
retrieving revision 1.226
diff -u -p -b -r1.226 mercury_wrapper.c
--- runtime/mercury_wrapper.c	13 Oct 2011 02:42:21 -0000	1.226
+++ runtime/mercury_wrapper.c	14 Nov 2011 06:33:45 -0000
@@ -352,7 +352,6 @@ int                 mercury_exit_status 
 
 MR_bool             MR_profiling = MR_TRUE;
 MR_bool             MR_print_deep_profiling_statistics = MR_FALSE;
-MR_bool             MR_output_deep_procrep_file;
 static  unsigned    MR_deep_prof_random_write = 0;
 static  MR_bool     MR_deep_profiling_save_results = MR_TRUE;
 static  MR_bool     MR_complexity_save_results = MR_TRUE;
@@ -1325,7 +1324,6 @@ enum MR_long_option {
     MR_NUM_OUTPUT_ARGS,
     MR_DEBUG_THREADS_OPT,
     MR_DEEP_PROF_DEBUG_FILE_OPT,
-    MR_DEEP_PROF_PROCREP_FILE_OPT,
     MR_DEEP_PROF_RANDOM_WRITE,
     MR_DEEP_PROF_LOG_FILE_OPT,
     MR_DEEP_PROF_LOG_PROG_OPT,
@@ -1427,7 +1425,6 @@ struct MR_option MR_long_opts[] = {
     { "num-output-args",                1, 0, MR_NUM_OUTPUT_ARGS },
     { "debug-threads",                  0, 0, MR_DEBUG_THREADS_OPT },
     { "deep-debug-file",                0, 0, MR_DEEP_PROF_DEBUG_FILE_OPT },
-    { "deep-procrep-file",              0, 0, MR_DEEP_PROF_PROCREP_FILE_OPT },
     /*
     ** The --deep-random-write option is only for use by tools/bootcheck.
     ** It is deliberately not documented.
@@ -1934,10 +1931,6 @@ MR_process_options(int argc, char **argv
                 MR_deep_prof_debug_file_flag = MR_TRUE;
                 break;
 
-            case MR_DEEP_PROF_PROCREP_FILE_OPT:
-                MR_output_deep_procrep_file = MR_TRUE;
-                break;
-
             case MR_DEEP_PROF_RANDOM_WRITE:
                 if (sscanf(MR_optarg, "%u", &MR_deep_prof_random_write) != 1) {
                     MR_usage();
Index: runtime/mercury_wrapper.h
===================================================================
RCS file: /home/mercury1/repository/mercury/runtime/mercury_wrapper.h,v
retrieving revision 1.88
diff -u -p -b -r1.88 mercury_wrapper.h
--- runtime/mercury_wrapper.h	27 Sep 2011 00:49:27 -0000	1.88
+++ runtime/mercury_wrapper.h	14 Nov 2011 06:33:45 -0000
@@ -339,7 +339,6 @@ extern	enum MR_TimeProfileMethod
 			MR_time_profile_method;
 
 extern	MR_bool		MR_profiling;
-extern	MR_bool		MR_output_deep_procrep_file;
 extern	MR_bool		MR_print_deep_profiling_statistics;
 
 #ifdef  MR_TYPE_CTOR_STATS
Index: tools/bootcheck
===================================================================
RCS file: /home/mercury1/repository/mercury/tools/bootcheck,v
retrieving revision 1.212
diff -u -p -b -r1.212 bootcheck
--- tools/bootcheck	8 Nov 2011 02:22:42 -0000	1.212
+++ tools/bootcheck	14 Nov 2011 06:33:45 -0000
@@ -583,8 +583,6 @@ then
     MERCURY_OPTIONS="$MERCURY_OPTIONS --deep-debug-file"
 fi
 
-MERCURY_OPTIONS="$MERCURY_OPTIONS --deep-procrep-file"
-
 if $trace_count
 then
     MERCURY_OPTIONS="$MERCURY_OPTIONS --trace-count-if-exec=mercury_compile"
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 489 bytes
Desc: Digital signature
URL: <http://lists.mercurylang.org/archives/reviews/attachments/20111114/3a94a7f4/attachment.sig>


More information about the reviews mailing list