[m-rev.] for review: compiler support for rbmm

Zoltan Somogyi zs at csse.unimelb.edu.au
Tue Jul 31 11:55:15 AEST 2007


On 25-Jul-2007, Zoltan Somogyi <zs at csse.unimelb.edu.au> wrote:
> The changes to ite_gen, disj_gen, code_info, llds and llds_out are for review
> by Quan, but I would like someone to review the rest of the diff as well
> (which will require reviewing the change to llds as well).

Quan identified an issue that required some significant changes in detail,
though not in concept. Here is the updated diff that I am committing.

Zoltan.

---------------------------------------------------------------------------

Add a first draft of the code generator support for region based memory
management. It is known to be incomplete; the missing parts are marked by XXXs.
It may also be buggy; it will be tested after Quan adds the runtime support,
i.e. the C macros invoked by the new LLDS instructions. However, the changes
in this diff shouldn't affect non-RBMM operations.

compiler/llds.m:
	Add five new LLDS instructions. Four are specific to RBMM operations.
	RBMM embeds three new stacks in compiler-reserved temp slots in
	procedure's usual Mercury stack frames, and the new LLDS instructions
	respectively
	
	(i)   push those stack frames onto their respective stacks,
	(ii)  fill some variable parts of those stack frames,
	(iii) fill fixed slots of those stack frames, and
	(iv)  use the contents of and/or pop those stack frames.

	(The pushing and popping affect only the new embedded stacks, not the
	usual Mercury stacks.)

	The last instruction is a new variant of the old assign instruction.
	It has identical semantics, but restricts optimization. An assign

	(a) can be deleted if its target lval is not used, and
	(b) its target lval can be changed (e.g. to a temp register) as long as
	    all the later instructions referring to that lval are changed to
	    use the new lval instead.

	Neither is permitted for the new keep_assign instruction. This is
	required because in an earlier draft we used it to assign to stack
	variables (parts of the embedded stack frames) that aren't explicitly
	referred to in later LLDS code, but are nevertheless implicitly
	referred to by some instructions (specifically iv above). We now
	use a specialized instruction (iii above) for this (since the macro
	it invokes can refer to C structure names, this makes it easier to
	keep the compiler in sync with the runtime system), but given that
	keep_assign is already implemented, may be useful later and shouldn't
	cause appreciable slowdown of the compiler, this diff keeps it.

	Extend the type that describe the contents of lvals to allow it
	to describe the new kinds of things we can now store in them.

	Add types to manage and describe the new embedded stack frames,
	and some utility functions. Change some existing utility functions
	to make all this more conceptually consistent.

compiler/ite_gen.m:
	Surround the code we generate for the condition of if-then-elses
	with the code required to ensure that regions that are logically
	removed in the condition aren't physically destroyed until we know
	that the condition succeeds (since the region may still be needed
	in the else branch), and to make sure that if the condition fails,
	all the memory allocated since the entry into the condition is
	reclaimed instantly.

compiler/disj_gen.m:
	Surround the code we generate for disjunctions with the code required
	to ensure that regions that are logically removed in a disjunct
	aren't physically destroyed if a later disjunct needs them, and to
	make sure that at entry into a non-first disjunct, all the memory
	allocated since the entry into the disjunction is reclaimed instantly.

compiler/commit_gen.m:
compiler/code_info.m:
	The protection against destruction offered by a disjunction disappears
	when a commit cuts away all later alternatives in that disjunct, so we
	must undo that protection. We therefore surround the scope of a commit
	goal with goal that achieves that objective.

	Add some new utility predicates to code_info. Remove some old utility
	functions that are now in llds.m.

compiler/continuation_info.m:
	Extend the type that describe the contents of stack slots to allow it
	to describe the new kinds of things we can now store in them.

	Rename the function symbols of that type to eliminate some ambiguities.

compiler/code_gen.m:
	Remember the set of variables live at the start of the goal
	(before the pre_goal_update updates it), since the region operations
	need to know this.

	Leave the lookup of AddTrailOps (and now AddRegionOps) to the specific
	kinds of goals that need it (the most frequent goals, unify and call,
	do not). Make both AddTrailOps and AddRegionOps use a self-explanatory
	type instead of a boolean.

compiler/lookup_switch.m:
	Conform to the change to AddTrailOps.

	Fix some misleading variable names.

compiler/options.m:
	Add some options to control the number of stack slots needed for
	various purposes. These have to correspond to the sizes of some C
	structures in the runtime system. Eventually these will be constants,
	but it is handy to keep them easily changeable while the C data
	structures are still being worked on.

	Add an option for optimizing away region ops whereever possible.
	The intention is that these should be on all the time, but we
	will want to turn them off for benchmarking.

compiler/dupelim.m:
compiler/dupproc.m:
compiler/exprn_aux.m:
compiler/frameopt.m:
compiler/global_data.m:
compiler/jumpopt.m:
compiler/livemap.m:
compiler/llds_out.m:
compiler/llds_to_x86_64.m:
compiler/middle_rec.m:
compiler/opt_debug.m:
compiler/opt_util.m:
compiler/par_conj_gen.m:
compiler/reassign.m:
compiler/stack_layout.m:
compiler/stdlabel.m:
compiler/trace_gen.m:
compiler/use_local_vars.m:
	Conform to the changes above, which mostly means handling the new
	LLDS instructions.

	In some cases, factor out existing common code, turn if-then-elses
	into switches, group common cases in switches, rationalize argument
	orders or variable names, and/or put code in execution order.

	In reassign.m, fix some old oversights that could (in some unlikely
	cases) cause bugs in the generated code.

compiler/pragma_c_gen.m:
	Exploit the capabilities of code_info.m.

compiler/prog_type.m:
	Add a utility predicate.

cvs diff: Diffing .
cvs diff: Diffing analysis
cvs diff: Diffing bindist
cvs diff: Diffing boehm_gc
cvs diff: Diffing boehm_gc/Mac_files
cvs diff: Diffing boehm_gc/cord
cvs diff: Diffing boehm_gc/cord/private
cvs diff: Diffing boehm_gc/doc
cvs diff: Diffing boehm_gc/include
cvs diff: Diffing boehm_gc/include/private
cvs diff: Diffing boehm_gc/libatomic_ops-1.2
cvs diff: Diffing boehm_gc/libatomic_ops-1.2/doc
cvs diff: Diffing boehm_gc/libatomic_ops-1.2/src
cvs diff: Diffing boehm_gc/libatomic_ops-1.2/src/atomic_ops
cvs diff: Diffing boehm_gc/libatomic_ops-1.2/src/atomic_ops/sysdeps
cvs diff: Diffing boehm_gc/libatomic_ops-1.2/src/atomic_ops/sysdeps/gcc
cvs diff: Diffing boehm_gc/libatomic_ops-1.2/src/atomic_ops/sysdeps/hpc
cvs diff: Diffing boehm_gc/libatomic_ops-1.2/src/atomic_ops/sysdeps/ibmc
cvs diff: Diffing boehm_gc/libatomic_ops-1.2/src/atomic_ops/sysdeps/icc
cvs diff: Diffing boehm_gc/libatomic_ops-1.2/src/atomic_ops/sysdeps/msftc
cvs diff: Diffing boehm_gc/libatomic_ops-1.2/src/atomic_ops/sysdeps/sunc
cvs diff: Diffing boehm_gc/libatomic_ops-1.2/tests
cvs diff: Diffing boehm_gc/tests
cvs diff: Diffing boehm_gc/windows-untested
cvs diff: Diffing boehm_gc/windows-untested/vc60
cvs diff: Diffing boehm_gc/windows-untested/vc70
cvs diff: Diffing boehm_gc/windows-untested/vc71
cvs diff: Diffing browser
cvs diff: Diffing bytecode
cvs diff: Diffing compiler
Index: compiler/code_gen.m
===================================================================
RCS file: /home/mercury/mercury1/repository/mercury/compiler/code_gen.m,v
retrieving revision 1.171
diff -u -b -r1.171 code_gen.m
--- compiler/code_gen.m	14 Jul 2007 02:32:41 -0000	1.171
+++ compiler/code_gen.m	23 Jul 2007 15:15:47 -0000
@@ -71,7 +71,9 @@
     % the generic data structures before and after the actual code generation,
     % which is delegated to goal-specific predicates.
 
-    % Make any changes to liveness before Goal
+    code_info.get_forward_live_vars(!.CI, ForwardLiveVarsBeforeGoal),
+
+    % Make any changes to liveness before Goal.
     ( goal_is_atomic(GoalExpr) ->
         IsAtomic = yes
     ;
@@ -101,7 +103,8 @@
             )
         ),
 
-        generate_goal_2(GoalExpr, GoalInfo, CodeModel, GoalCode, !CI),
+        generate_goal_2(GoalExpr, GoalInfo, CodeModel,
+            ForwardLiveVarsBeforeGoal, GoalCode, !CI),
         goal_info_get_features(GoalInfo, Features),
         code_info.get_proc_info(!.CI, ProcInfo),
 
@@ -192,9 +195,11 @@
 %---------------------------------------------------------------------------%
 
 :- pred generate_goal_2(hlds_goal_expr::in, hlds_goal_info::in,
-    code_model::in, code_tree::out, code_info::in, code_info::out) is det.
+    code_model::in, set(prog_var)::in, code_tree::out,
+    code_info::in, code_info::out) is det.
 
-generate_goal_2(GoalExpr, GoalInfo, CodeModel, Code, !CI) :-
+generate_goal_2(GoalExpr, GoalInfo, CodeModel, ForwardLiveVarsBeforeGoal,
+        Code, !CI) :-
     (
         GoalExpr = unify(_, _, _, Uni, _),
         unify_gen.generate_unification(CodeModel, Uni, GoalInfo, Code, !CI)
@@ -210,38 +215,35 @@
         )
     ;
         GoalExpr = disj(Goals),
-        AddTrailOps = should_add_trail_ops(!.CI, GoalInfo),
-        disj_gen.generate_disj(AddTrailOps, CodeModel, Goals, GoalInfo, Code,
-            !CI)
+        disj_gen.generate_disj(CodeModel, Goals, GoalInfo, Code, !CI)
     ;
         GoalExpr = negation(Goal),
-        AddTrailOps = should_add_trail_ops(!.CI, GoalInfo),
-        ite_gen.generate_negation(AddTrailOps, CodeModel, Goal, GoalInfo,
-            Code, !CI)
+        ite_gen.generate_negation(CodeModel, Goal, GoalInfo, Code, !CI)
     ;
         GoalExpr = if_then_else(_Vars, Cond, Then, Else),
-        AddTrailOps = should_add_trail_ops(!.CI, GoalInfo),
-        ite_gen.generate_ite(AddTrailOps, CodeModel, Cond, Then, Else,
-            GoalInfo, Code, !CI)
+        ite_gen.generate_ite(CodeModel, Cond, Then, Else, GoalInfo, Code, !CI)
     ;
         GoalExpr = switch(Var, CanFail, CaseList),
         switch_gen.generate_switch(CodeModel, Var, CanFail, CaseList, GoalInfo,
             Code, !CI)
     ;
         GoalExpr = scope(Reason, Goal),
-        AddTrailOps = should_add_trail_ops(!.CI, GoalInfo),
-        commit_gen.generate_scope(Reason, AddTrailOps, CodeModel, Goal, Code,
-            !CI)
+        commit_gen.generate_scope(Reason, CodeModel, GoalInfo,
+            ForwardLiveVarsBeforeGoal, Goal, Code, !CI)
     ;
         GoalExpr = generic_call(GenericCall, Args, Modes, Det),
         call_gen.generate_generic_call(CodeModel, GenericCall, Args,
             Modes, Det, GoalInfo, Code, !CI)
     ;
         GoalExpr = plain_call(PredId, ProcId, Args, BuiltinState, _, _),
-        ( BuiltinState = not_builtin ->
-            call_gen.generate_call(CodeModel, PredId, ProcId, Args,
-                GoalInfo, Code, !CI)
+        (
+            BuiltinState = not_builtin,
+            call_gen.generate_call(CodeModel, PredId, ProcId, Args, GoalInfo,
+                Code, !CI)
         ;
+            ( BuiltinState = inline_builtin
+            ; BuiltinState = out_of_line_builtin
+            ),
             call_gen.generate_builtin(CodeModel, PredId, ProcId, Args,
                 Code, !CI)
         )
@@ -249,7 +251,8 @@
         GoalExpr = call_foreign_proc(Attributes, PredId, ProcId,
             Args, ExtraArgs, MaybeTraceRuntimeCond, PragmaCode),
         Lang = get_foreign_language(Attributes),
-        (   Lang = lang_c,
+        (
+            Lang = lang_c,
             generate_foreign_proc_code(CodeModel, Attributes,
                 PredId, ProcId, Args, ExtraArgs, MaybeTraceRuntimeCond,
                 PragmaCode, GoalInfo, Code, !CI)
Index: compiler/code_info.m
===================================================================
RCS file: /home/mercury/mercury1/repository/mercury/compiler/code_info.m,v
retrieving revision 1.346
diff -u -b -r1.346 code_info.m
--- compiler/code_info.m	13 Jul 2007 03:27:26 -0000	1.346
+++ compiler/code_info.m	30 Jul 2007 08:22:56 -0000
@@ -143,6 +143,8 @@
 
 :- pred get_emit_trail_ops(code_info::in, add_trail_ops::out) is det.
 
+:- pred get_emit_region_ops(code_info::in, add_region_ops::out) is det.
+
     % Get the set of currently forward-live variables.
     %
 :- pred get_forward_live_vars(code_info::in, set(prog_var)::out) is det.
@@ -214,6 +216,8 @@
 
 :- pred get_opt_trail_ops(code_info::in, bool::out) is det.
 
+:- pred get_opt_region_ops(code_info::in, bool::out) is det.
+
 :- pred get_auto_comments(code_info::in, bool::out) is det.
 
 :- pred get_lcmc_null(code_info::in, bool::out) is det.
@@ -343,13 +347,20 @@
                                     % Should we optimize calls that cannot
                                     % return?
 
-                emit_trail_ops      :: bool,
+                emit_trail_ops      :: add_trail_ops,
                                     % Should we emit trail operations?
 
                 opt_trail_ops       :: bool,
                                     % Should we try to avoid emiting trail
                                     % operations?
 
+                emit_region_ops     :: add_region_ops,
+                                    % Should we emit region operations?
+
+                opt_region_ops      :: bool,
+                                    % Should we try to avoid emiting region
+                                    % operations?
+
                 auto_comments       :: bool,
                                     % The setting of --auto-comments.
 
@@ -517,11 +528,20 @@
         UseTrail = yes,
         DisableTrailOps = no
     ->
-        EmitTrailOps = yes
+        EmitTrailOps = add_trail_ops
     ;
-        EmitTrailOps = no
+        EmitTrailOps = do_not_add_trail_ops
     ),
     globals.lookup_bool_option(Globals, optimize_trail_usage, OptTrailOps),
+    globals.lookup_bool_option(Globals, optimize_region_ops, OptRegionOps),
+    globals.lookup_bool_option(Globals, use_regions, UseRegions),
+    (
+        UseRegions = yes,
+        EmitRegionOps = add_region_ops
+    ;
+        UseRegions = no,
+        EmitRegionOps = do_not_add_region_ops
+    ),
     globals.lookup_bool_option(Globals, auto_comments, AutoComments),
     globals.lookup_bool_option(Globals, optimize_constructor_last_call_null,
         LCMCNull),
@@ -539,6 +559,8 @@
             OptNoReturnCalls,
             EmitTrailOps,
             OptTrailOps,
+            EmitRegionOps,
+            OptRegionOps,
             AutoComments,
             LCMCNull
         ),
@@ -603,6 +625,8 @@
 get_opt_no_return_calls(CI, CI ^ code_info_static ^ opt_no_resume_calls).
 get_emit_trail_ops(CI, CI ^ code_info_static ^ emit_trail_ops).
 get_opt_trail_ops(CI, CI ^ code_info_static ^ opt_trail_ops).
+get_emit_region_ops(CI, CI ^ code_info_static ^ emit_region_ops).
+get_opt_region_ops(CI, CI ^ code_info_static ^ opt_region_ops).
 get_auto_comments(CI, CI ^ code_info_static ^ auto_comments).
 get_lcmc_null(CI, CI ^ code_info_static ^ lcmc_null).
 get_forward_live_vars(CI, CI ^ code_info_loc_dep ^ forward_live_vars).
@@ -708,6 +732,8 @@
 
 :- func lookup_type_defn(code_info, mer_type) = hlds_type_defn.
 
+:- func filter_region_vars(code_info, set(prog_var)) = set(prog_var).
+
     % Given a constructor id, and a variable (so that we can work out the
     % type of the constructor), determine correct tag (representation)
     % of that constructor.
@@ -888,6 +914,11 @@
     module_info_get_type_table(ModuleInfo, TypeTable),
     map.lookup(TypeTable, TypeCtor, TypeDefn).
 
+filter_region_vars(CI, ForwardLiveVarsBeforeGoal) = RegionVars :-
+    VarTypes = code_info.get_var_types(CI),
+    RegionVars = set.filter(is_region_var(VarTypes),
+        ForwardLiveVarsBeforeGoal).
+
 cons_id_to_tag_for_var(CI, Var, ConsId) = ConsTag :-
     get_module_info(CI, ModuleInfo),
     ConsTag = cons_id_to_tag(ConsId, variable_type(CI, Var), ModuleInfo).
@@ -1302,10 +1333,14 @@
     % being cut across. If the goal succeeds, the commit will cut
     % any choice points generated in the goal.
     %
+    % The set(prog_var) should be the set of variables live before
+    % the scope goal.
+    %
 :- type det_commit_info.
 
-:- pred prepare_for_det_commit(add_trail_ops::in, det_commit_info::out,
-    code_tree::out, code_info::in, code_info::out) is det.
+:- pred prepare_for_det_commit(add_trail_ops::in, add_region_ops::in,
+    set(prog_var)::in, det_commit_info::out, code_tree::out,
+    code_info::in, code_info::out) is det.
 
 :- pred generate_det_commit(det_commit_info::in,
     code_tree::out, code_info::in, code_info::out) is det.
@@ -1315,10 +1350,14 @@
     % being cut across. If the goal succeeds, the commit will cut
     % any choice points generated in the goal.
     %
+    % The set(prog_var) should be the set of variables live before
+    % the scope goal.
+    %
 :- type semi_commit_info.
 
-:- pred prepare_for_semi_commit(bool::in, semi_commit_info::out,
-    code_tree::out, code_info::in, code_info::out) is det.
+:- pred prepare_for_semi_commit(add_trail_ops::in, add_region_ops::in,
+    set(prog_var)::in, semi_commit_info::out, code_tree::out,
+    code_info::in, code_info::out) is det.
 
 :- pred generate_semi_commit(semi_commit_info::in,
     code_tree::out, code_info::in, code_info::out) is det.
@@ -1529,7 +1568,7 @@
     ->
         % Here ResumeKnown must be resume_point_unknown
         % or resume_point_known(wont_be_done).
-        acquire_temp_slot(lval(redoip_slot(lval(curfr))),
+        acquire_temp_slot(slot_lval(redoip_slot(lval(curfr))),
             non_persistent_temp_slot, RedoipSlot, !CI),
         HijackInfo = disj_half_hijack(RedoipSlot),
         Code = node([
@@ -1538,9 +1577,9 @@
         ])
     ;
         % Here CurfrMaxfr must be may_be_different.
-        acquire_temp_slot(lval(redoip_slot(lval(maxfr))),
+        acquire_temp_slot(slot_lval(redoip_slot(lval(maxfr))),
             non_persistent_temp_slot, RedoipSlot, !CI),
-        acquire_temp_slot(lval(redofr_slot(lval(maxfr))),
+        acquire_temp_slot(slot_lval(redofr_slot(lval(maxfr))),
             non_persistent_temp_slot, RedofrSlot, !CI),
         HijackInfo = disj_full_hijack(RedoipSlot, RedofrSlot),
         Code = node([
@@ -1655,8 +1694,8 @@
     ;
         ( Allow = not_allowed ; CondEnv = inside_non_condition )
     ->
-        acquire_temp_slot(lval(maxfr), non_persistent_temp_slot, MaxfrSlot,
-            !CI),
+        acquire_temp_slot(slot_lval(maxfr), non_persistent_temp_slot,
+            MaxfrSlot, !CI),
         HijackType = ite_temp_frame(MaxfrSlot),
         create_temp_frame(do_fail, "prepare for ite", TempFrameCode, !CI),
         MaxfrCode = node([
@@ -1675,7 +1714,7 @@
         CurfrMaxfr = must_be_equal
     ->
         % Here ResumeKnown must be resume_point_unknown.
-        acquire_temp_slot(lval(redoip_slot(lval(curfr))),
+        acquire_temp_slot(slot_lval(redoip_slot(lval(curfr))),
             non_persistent_temp_slot, RedoipSlot, !CI),
         HijackType = ite_half_hijack(RedoipSlot),
         Code = node([
@@ -1684,11 +1723,11 @@
         ])
     ;
         % Here CurfrMaxfr must be may_be_different.
-        acquire_temp_slot(lval(redoip_slot(lval(maxfr))),
+        acquire_temp_slot(slot_lval(redoip_slot(lval(maxfr))),
             non_persistent_temp_slot, RedoipSlot, !CI),
-        acquire_temp_slot(lval(redofr_slot(lval(maxfr))),
+        acquire_temp_slot(slot_lval(redofr_slot(lval(maxfr))),
             non_persistent_temp_slot, RedofrSlot, !CI),
-        acquire_temp_slot(lval(maxfr),
+        acquire_temp_slot(slot_lval(maxfr),
             non_persistent_temp_slot, MaxfrSlot, !CI),
         HijackType = ite_full_hijack(RedoipSlot, RedofrSlot, MaxfrSlot),
         Code = node([
@@ -1727,7 +1766,7 @@
                 "soft cut for temp frame ite")
         ]),
         ElseCode = node([
-            % XXX, search /assign(maxfr
+            % XXX search for assignments to maxfr
             llds_instr(assign(maxfr, lval(prevfr_slot(lval(MaxfrSlot)))),
                 "restore maxfr for temp frame ite")
         ])
@@ -1767,9 +1806,11 @@
                 "restore redofr for full ite hijack")
         ])
     ),
-    ( ResumeKnown0 = resume_point_unknown ->
+    (
+        ResumeKnown0 = resume_point_unknown,
         ResumeKnown = resume_point_unknown
     ;
+        ResumeKnown0 = resume_point_known(_),
         ResumeKnown = HijackResumeKnown
     ),
     FailInfo = fail_info(ResumePoints, ResumeKnown, CurfrMaxfr, OldCondEnv,
@@ -1816,17 +1857,37 @@
 :- type det_commit_info
     --->    det_commit_info(
                 maybe(lval),        % Location of saved maxfr.
-                maybe(pair(lval))   % Location of saved ticket
+                maybe(pair(lval)),  % Location of saved ticket
                                     % counter and trail pointer.
+                maybe(region_commit_stack_frame)
+            ).
+
+:- type region_commit_stack_frame
+    --->    region_commit_stack_frame(
+                embedded_stack_frame_id,
+                                    % The id of the region commit stack frame,
+                                    % which is emdedded in the current
+                                    % procedure's stack frame, and whose
+                                    % layout is:
+
+                                    % saved region_commit_stack_pointer
+                                    % saved region sequence number
+                                    % number of live nonprotected regions
+                                    % space reserved for the ids of live
+                                    %   nonprotected regions
+
+                list(lval)          % The list of temporary slots that
+                                    % constitute this embedded stack frame.
             ).
 
-prepare_for_det_commit(AddTrailOps, DetCommitInfo, Code, !CI) :-
+prepare_for_det_commit(AddTrailOps, AddRegionOps, ForwardLiveVarsBeforeGoal,
+        DetCommitInfo, Code, !CI) :-
     get_fail_info(!.CI, FailInfo0),
     FailInfo0 = fail_info(_, _, CurfrMaxfr, _, _),
     (
         CurfrMaxfr = may_be_different,
-        acquire_temp_slot(lval(maxfr), non_persistent_temp_slot, MaxfrSlot,
-            !CI),
+        acquire_temp_slot(slot_lval(maxfr), non_persistent_temp_slot,
+            MaxfrSlot, !CI),
         SaveMaxfrCode = node([
             llds_instr(save_maxfr(MaxfrSlot), "save the value of maxfr")
         ]),
@@ -1837,11 +1898,19 @@
         MaybeMaxfrSlot = no
     ),
     maybe_save_trail_info(AddTrailOps, MaybeTrailSlots, SaveTrailCode, !CI),
-    DetCommitInfo = det_commit_info(MaybeMaxfrSlot, MaybeTrailSlots),
-    Code = tree(SaveMaxfrCode, SaveTrailCode).
+    maybe_save_region_commit_frame(AddRegionOps, ForwardLiveVarsBeforeGoal,
+        MaybeRegionCommitFrameInfo, SaveRegionCommitFrameCode, !CI),
+    DetCommitInfo = det_commit_info(MaybeMaxfrSlot, MaybeTrailSlots,
+        MaybeRegionCommitFrameInfo),
+    Code = tree_list([
+        SaveMaxfrCode,
+        SaveTrailCode,
+        SaveRegionCommitFrameCode
+    ]).
 
 generate_det_commit(DetCommitInfo, Code, !CI) :-
-    DetCommitInfo = det_commit_info(MaybeMaxfrSlot, MaybeTrailSlots),
+    DetCommitInfo = det_commit_info(MaybeMaxfrSlot, MaybeTrailSlots,
+        MaybeRegionCommitFrameInfo),
     (
         MaybeMaxfrSlot = yes(MaxfrSlot),
         RestoreMaxfrCode = node([
@@ -1857,7 +1926,13 @@
         ])
     ),
     maybe_restore_trail_info(MaybeTrailSlots, CommitTrailCode, _, !CI),
-    Code = tree(RestoreMaxfrCode, CommitTrailCode).
+    maybe_restore_region_commit_frame(MaybeRegionCommitFrameInfo,
+        SuccessRegionCode, _FailureRegionCode, !CI),
+    Code = tree_list([
+        RestoreMaxfrCode,
+        CommitTrailCode,
+        SuccessRegionCode
+    ]).
 
 %---------------------------------------------------------------------------%
 
@@ -1866,8 +1941,9 @@
                 fail_info,              % Fail_info on entry.
                 resume_point_info,
                 commit_hijack_info,
-                maybe(pair(lval))       % Location of saved ticket
+                maybe(pair(lval)),      % Location of saved ticket
                                         % counter and trail pointer.
+                maybe(region_commit_stack_frame)
             ).
 
 :- type commit_hijack_info
@@ -1891,7 +1967,8 @@
                             % the value of maxfr.
             ).
 
-prepare_for_semi_commit(AddTrailOps, SemiCommitInfo, Code, !CI) :-
+prepare_for_semi_commit(AddTrailOps, AddRegionOps, ForwardLiveVarsBeforeGoal,
+        SemiCommitInfo, Code, !CI) :-
     get_fail_info(!.CI, FailInfo0),
     FailInfo0 = fail_info(ResumePoints0, ResumeKnown, CurfrMaxfr, CondEnv,
         Allow),
@@ -1907,8 +1984,8 @@
     (
         ( Allow = not_allowed ; CondEnv = inside_non_condition )
     ->
-        acquire_temp_slot(lval(maxfr), non_persistent_temp_slot, MaxfrSlot,
-            !CI),
+        acquire_temp_slot(slot_lval(maxfr), non_persistent_temp_slot,
+            MaxfrSlot, !CI),
         MaxfrCode = node([
             llds_instr(save_maxfr(MaxfrSlot),
                 "prepare for temp frame commit")
@@ -1921,15 +1998,13 @@
         HijackInfo = commit_temp_frame(MaxfrSlot, UseMinimalModelStackCopyCut),
         (
             UseMinimalModelStackCopyCut = yes,
-            % If the code we are committing across starts but
-            % does not complete the evaluation of a tabled subgoal,
-            % the cut will remove the generator's choice point,
-            % so that the evaluation of the subgoal will never
-            % be completed. We handle such "dangling" generators
-            % by removing them from the subgoal trie of the
-            % tabled procedure. This requires knowing what
-            % tabled subgoals are started inside commits,
-            % which is why we wrap the goal being committed across
+            % If the code we are committing across starts but does not complete
+            % the evaluation of a tabled subgoal, the cut will remove the
+            % generator's choice point, so that the evaluation of the subgoal
+            % will never be completed. We handle such "dangling" generators
+            % by removing them from the subgoal trie of the tabled procedure.
+            % This requires knowing what tabled subgoals are started inside
+            % commits, which is why we wrap the goal being committed across
             % inside MR_commit_{mark,cut}.
             Components = [
                 foreign_proc_raw_code(cannot_branch_away,
@@ -1967,7 +2042,7 @@
         % Here ResumeKnown must be resume_point_unknown or
         % resume_point_known(wont_be_done).
 
-        acquire_temp_slot(lval(redoip_slot(lval(curfr))),
+        acquire_temp_slot(slot_lval(redoip_slot(lval(curfr))),
             non_persistent_temp_slot, RedoipSlot, !CI),
         HijackInfo = commit_half_hijack(RedoipSlot),
         HijackCode = node([
@@ -1978,11 +2053,11 @@
         ])
     ;
         % Here CurfrMaxfr must be may_be_different.
-        acquire_temp_slot(lval(redoip_slot(lval(maxfr))),
+        acquire_temp_slot(slot_lval(redoip_slot(lval(maxfr))),
             non_persistent_temp_slot, RedoipSlot, !CI),
-        acquire_temp_slot(lval(redofr_slot(lval(maxfr))),
+        acquire_temp_slot(slot_lval(redofr_slot(lval(maxfr))),
             non_persistent_temp_slot, RedofrSlot, !CI),
-        acquire_temp_slot(lval(maxfr),
+        acquire_temp_slot(slot_lval(maxfr),
             non_persistent_temp_slot, MaxfrSlot, !CI),
         HijackInfo = commit_full_hijack(RedoipSlot, RedofrSlot, MaxfrSlot),
         HijackCode = node([
@@ -1999,13 +2074,19 @@
         ])
     ),
     maybe_save_trail_info(AddTrailOps, MaybeTrailSlots, SaveTrailCode, !CI),
+    maybe_save_region_commit_frame(AddRegionOps, ForwardLiveVarsBeforeGoal,
+        MaybeRegionCommitFrameInfo, SaveRegionCommitFrameCode, !CI),
     SemiCommitInfo = semi_commit_info(FailInfo0, NewResumePoint,
-        HijackInfo, MaybeTrailSlots),
-    Code = tree(HijackCode, SaveTrailCode).
+        HijackInfo, MaybeTrailSlots, MaybeRegionCommitFrameInfo),
+    Code = tree_list([
+        HijackCode,
+        SaveTrailCode,
+        SaveRegionCommitFrameCode
+    ]).
 
 generate_semi_commit(SemiCommitInfo, Code, !CI) :-
     SemiCommitInfo = semi_commit_info(FailInfo, ResumePoint,
-        HijackInfo, MaybeTrailSlots),
+        HijackInfo, MaybeTrailSlots, MaybeRegionCommitFrameInfo),
 
     set_fail_info(FailInfo, !CI),
     % XXX Should release the temp slots in each arm of the switch.
@@ -2088,6 +2169,8 @@
 
     maybe_restore_trail_info(MaybeTrailSlots, CommitTrailCode,
         RestoreTrailCode, !CI),
+    maybe_restore_region_commit_frame(MaybeRegionCommitFrameInfo,
+        SuccessRegionCode, FailureRegionCode, !CI),
 
     get_next_label(SuccLabel, !CI),
     GotoSuccLabel = node([
@@ -2096,11 +2179,144 @@
     SuccLabelCode = node([
         llds_instr(label(SuccLabel), "Success continuation")
     ]),
-    SuccessCode = tree(SuccessUndoCode, CommitTrailCode),
-    FailureCode = tree_list([ResumePointCode, FailureUndoCode,
-        RestoreTrailCode, FailCode]),
-    Code = tree_list([SuccessCode, GotoSuccLabel,
-        FailureCode, SuccLabelCode]).
+    SuccessCode = tree_list([
+        SuccessUndoCode,
+        CommitTrailCode,
+        SuccessRegionCode
+    ]),
+    FailureCode = tree_list([
+        ResumePointCode,
+        FailureUndoCode,
+        RestoreTrailCode,
+        FailureRegionCode,
+        FailCode
+    ]),
+    Code = tree_list([
+        SuccessCode,
+        GotoSuccLabel,
+        FailureCode,
+        SuccLabelCode
+    ]).
+
+%---------------------------------------------------------------------------%
+
+:- pred maybe_save_region_commit_frame(add_region_ops::in, set(prog_var)::in,
+    maybe(region_commit_stack_frame)::out, code_tree::out,
+    code_info::in, code_info::out) is det.
+
+maybe_save_region_commit_frame(AddRegionOps, ForwardLiveVarsBeforeGoal,
+        MaybeRegionCommitFrameInfo, Code, !CI) :-
+    (
+        AddRegionOps = do_not_add_region_ops,
+        MaybeRegionCommitFrameInfo = no,
+        Code = empty
+    ;
+        AddRegionOps = add_region_ops,
+        RegionVars = filter_region_vars(!.CI, ForwardLiveVarsBeforeGoal),
+
+        % XXX RemovedRegionVars should be the set of region vars whose
+        % regions are removed in the scope. However, this information
+        % is not yet available in the HLDS, and there is no simple way here
+        % to reconstruct it.
+        RemovedRegionVars = RegionVars,
+
+        RemovedRegionVarList = set.to_sorted_list(RemovedRegionVars),
+
+        NumRemovedRegionVars = list.length(RemovedRegionVarList),
+
+        code_info.get_globals(!.CI, Globals),
+        globals.lookup_int_option(Globals, size_region_commit_fixed,
+            FixedSize),
+        globals.lookup_int_option(Globals, size_region_commit_entry,
+            EntrySize),
+        FrameSize = FixedSize + EntrySize * NumRemovedRegionVars,
+        Items = list.duplicate(FrameSize, slot_region_commit),
+        acquire_several_temp_slots(Items, non_persistent_temp_slot,
+            StackVars, MainStackId, FirstSlotNum, LastSlotNum, !CI),
+        EmbeddedStackFrame = embedded_stack_frame_id(MainStackId,
+            FirstSlotNum, LastSlotNum),
+        FirstSavedRegionAddr =
+            first_nonfixed_embedded_slot_addr(EmbeddedStackFrame, FixedSize),
+        acquire_reg(reg_r, NumRegLval, !CI),
+        acquire_reg(reg_r, AddrRegLval, !CI),
+        PushInitCode = node([
+            llds_instr(
+                push_region_frame(region_stack_commit, EmbeddedStackFrame),
+                "Save stack pointer of embedded region commit stack"),
+            llds_instr(
+                assign(NumRegLval, const(llconst_int(0))),
+                "Initialize number of unprotected live regions"),
+            llds_instr(
+                assign(AddrRegLval, FirstSavedRegionAddr),
+                "Initialize pointer to the next unprotected live region slot")
+        ]),
+        save_unprotected_live_regions(NumRegLval, AddrRegLval,
+            EmbeddedStackFrame, RemovedRegionVarList, FillCode, !CI),
+        SetCode = node([
+            llds_instr(
+                region_set_fixed_slot(region_set_commit_num_entries,
+                    EmbeddedStackFrame, lval(NumRegLval)),
+                "Store the number of unprotected live regions")
+        ]),
+        release_reg(NumRegLval, !CI),
+        release_reg(AddrRegLval, !CI),
+
+        RegionCommitFrameInfo = region_commit_stack_frame(EmbeddedStackFrame,
+            StackVars),
+        MaybeRegionCommitFrameInfo = yes(RegionCommitFrameInfo),
+
+        Code = tree_list([
+            PushInitCode,
+            FillCode,
+            SetCode
+        ])
+    ).
+
+:- pred save_unprotected_live_regions(lval::in, lval::in,
+    embedded_stack_frame_id::in, list(prog_var)::in, code_tree::out,
+    code_info::in, code_info::out) is det.
+
+save_unprotected_live_regions(_, _, _, [], empty, !CI).
+save_unprotected_live_regions(NumLval, AddrLval, EmbeddedStackFrame,
+        [RegionVar | RegionVars], tree(Code, Codes), !CI) :-
+    produce_variable(RegionVar, ProduceVarCode, RegionVarRval, !CI),
+    SaveCode = node([
+        llds_instr(
+            region_fill_frame(region_fill_commit, EmbeddedStackFrame,
+                RegionVarRval, NumLval, AddrLval),
+            "Save the region in the commit stack frame if it is unprotected")
+    ]),
+    Code = tree(ProduceVarCode, SaveCode),
+    save_unprotected_live_regions(NumLval, AddrLval, EmbeddedStackFrame,
+        RegionVars, Codes, !CI).
+
+:- pred maybe_restore_region_commit_frame(maybe(region_commit_stack_frame)::in,
+    code_tree::out, code_tree::out, code_info::in, code_info::out) is det.
+
+maybe_restore_region_commit_frame(MaybeRegionCommitFrameInfo,
+        SuccessCode, FailureCode, !CI) :-
+    (
+        MaybeRegionCommitFrameInfo = no,
+        SuccessCode = empty,
+        FailureCode = empty
+    ;
+        MaybeRegionCommitFrameInfo = yes(RegionCommitFrameInfo),
+        RegionCommitFrameInfo = region_commit_stack_frame(EmbeddedStackFrame,
+            StackVars),
+        SuccessCode = node([
+            llds_instr(
+                use_and_maybe_pop_region_frame(region_commit_success,
+                    EmbeddedStackFrame),
+                "Destroy removed regions protected by cut away disjunctions")
+        ]),
+        FailureCode = node([
+            llds_instr(
+                use_and_maybe_pop_region_frame(region_commit_failure,
+                    EmbeddedStackFrame),
+                "Undo the creation of the commit frame")
+        ]),
+        release_several_temp_slots(StackVars, non_persistent_temp_slot, !CI)
+    ).
 
 %---------------------------------------------------------------------------%
 
@@ -2668,15 +2884,15 @@
 
 %---------------------------------------------------------------------------%
 
-:- pred maybe_save_trail_info(bool::in, maybe(pair(lval))::out,
+:- pred maybe_save_trail_info(add_trail_ops::in, maybe(pair(lval))::out,
     code_tree::out, code_info::in, code_info::out) is det.
 
 maybe_save_trail_info(AddTrailOps, MaybeTrailSlots, SaveTrailCode, !CI) :-
     (
-        AddTrailOps = yes,
-        acquire_temp_slot(ticket_counter, non_persistent_temp_slot,
+        AddTrailOps = add_trail_ops,
+        acquire_temp_slot(slot_ticket_counter, non_persistent_temp_slot,
             CounterSlot, !CI),
-        acquire_temp_slot(ticket, non_persistent_temp_slot,
+        acquire_temp_slot(slot_ticket, non_persistent_temp_slot,
             TrailPtrSlot, !CI),
         MaybeTrailSlots = yes(CounterSlot - TrailPtrSlot),
         SaveTrailCode = node([
@@ -2686,7 +2902,7 @@
                 "save the trail pointer")
         ])
     ;
-        AddTrailOps = no,
+        AddTrailOps = do_not_add_trail_ops,
         MaybeTrailSlots = no,
         SaveTrailCode = empty
     ).
@@ -2917,7 +3133,7 @@
 :- pred discard_and_release_ticket(lval::in, code_tree::out,
     code_info::in, code_info::out) is det.
 
-:- pred maybe_save_ticket(bool::in, code_tree::out,
+:- pred maybe_save_ticket(add_trail_ops::in, code_tree::out,
     maybe(lval)::out, code_info::in, code_info::out) is det.
 
 :- pred maybe_reset_ticket(maybe(lval)::in, reset_trail_reason::in,
@@ -2948,12 +3164,17 @@
     %
 :- func should_add_trail_ops(code_info, hlds_goal_info) = add_trail_ops.
 
+    % Should we add region ops to the code we generate for the goal with the
+    % given goal_info.  This will be 'no' unless we are in a rbmm grade.
+    %
+:- func should_add_region_ops(code_info, hlds_goal_info) = add_region_ops.
+
 %---------------------------------------------------------------------------%
 
 :- implementation.
 
 save_hp(Code, HpSlot, !CI) :-
-    acquire_temp_slot(lval(hp), non_persistent_temp_slot, HpSlot, !CI),
+    acquire_temp_slot(slot_lval(hp), non_persistent_temp_slot, HpSlot, !CI),
     Code = node([
         llds_instr(mark_hp(HpSlot), "Save heap pointer")
     ]).
@@ -3012,7 +3233,7 @@
 %---------------------------------------------------------------------------%
 
 save_ticket(Code, TicketSlot, !CI) :-
-    acquire_temp_slot(ticket, non_persistent_temp_slot, TicketSlot, !CI),
+    acquire_temp_slot(slot_ticket, non_persistent_temp_slot, TicketSlot, !CI),
     Code = node([
         llds_instr(store_ticket(TicketSlot), "Save trail state")
     ]).
@@ -3059,13 +3280,13 @@
 
 %---------------------------------------------------------------------------%
 
-maybe_save_ticket(Maybe, Code, MaybeTicketSlot, !CI) :-
+maybe_save_ticket(AddTrailOps, Code, MaybeTicketSlot, !CI) :-
     (
-        Maybe = yes,
+        AddTrailOps = add_trail_ops,
         save_ticket(Code, TicketSlot, !CI),
         MaybeTicketSlot = yes(TicketSlot)
     ;
-        Maybe = no,
+        AddTrailOps = do_not_add_trail_ops,
         Code = empty,
         MaybeTicketSlot = no
     ).
@@ -3136,17 +3357,15 @@
         Code = empty
     ).
 
-    % XXX We will eventually need to pass GoalInfo here.
+    % XXX We will eventually need to make use of GoalInfo here.
     %
 should_add_trail_ops(CodeInfo, _GoalInfo) = AddTrailOps :-
-    get_emit_trail_ops(CodeInfo, EmitTrailOps),
-    (
-        EmitTrailOps = no,
-        AddTrailOps = no
-    ;
-        EmitTrailOps = yes,
-        AddTrailOps = yes
-    ).
+    get_emit_trail_ops(CodeInfo, AddTrailOps).
+
+    % XXX We will eventually need to make use of GoalInfo here.
+    %
+should_add_region_ops(CodeInfo, _GoalInfo) = AddRegionOps :-
+    get_emit_region_ops(CodeInfo, AddRegionOps).
 
 %---------------------------------------------------------------------------%
 %---------------------------------------------------------------------------%
@@ -3411,8 +3630,8 @@
         var_locn_acquire_reg_prefer_given(PrefRegNum, Lval,
         VarLocnInfo0, VarLocnInfo)
     ;
-        % XXX We should only get a register if the map.search
-        % succeeded; otherwise we should put the var in its stack slot.
+        % XXX We should only get a register if the map.search succeeded;
+        % otherwise we should put the var in its stack slot.
         var_locn_acquire_reg_start_at_given(NextNonReserved, Lval,
             VarLocnInfo0, VarLocnInfo)
     ),
@@ -3816,15 +4035,19 @@
 :- pred release_temp_slot(lval::in, temp_slot_persistence::in,
     code_info::in, code_info::out) is det.
 
+    % acquire_several_temp_slots(Items, Persistence, StackVars,
+    %   StackId, N, M, !Info):
+    %
     % Perform an acquire_temp_slot operation for each element of the
     % input list, all with the same persistence.
     %
-    % Given an input list of length N, the returned list will contain
-    % N consecutive stack slots.
+    % The slots will be the ones from stack_slot_num_to_lval(StackId, N)
+    % consecutively to stack_slot_num_to_lval(StackId, M), with N < M.
+    % These will also be returned as StackVars.
     %
 :- pred acquire_several_temp_slots(list(slot_contents)::in,
     temp_slot_persistence::in, list(lval)::out,
-    code_info::in, code_info::out) is det.
+    main_stack::out, int::out, int::out, code_info::in, code_info::out) is det.
 
     % Release the stack slots acquired by an earlier acquire_several_temp_slots
     % operation. The persistence argument should match the acquire operation.
@@ -3865,22 +4088,27 @@
         Persistence = non_persistent_temp_slot
     ).
 
-acquire_several_temp_slots([], _, _, !CI) :-
-    % We could return an empty list of stack vars, but currently,
-    % this is always an error.
+acquire_several_temp_slots([], _, _, _, _, _, !CI) :-
+    % We could return an empty list of stack vars for StackVars, but there is
+    % nothing meaningful we can return for the other outputs.
     unexpected(this_file, "acquire_several_temp_slots: []").
 acquire_several_temp_slots([HeadItem | TailItems], Persistence, StackVars,
-        !CI) :-
+        StackId, FirstSlotNum, LastSlotNum, !CI) :-
     get_temp_content_map(!.CI, TempContentMap0),
     map.to_assoc_list(TempContentMap0, TempContentList),
     get_temps_in_use(!.CI, TempsInUse0),
     (
         find_unused_slots_for_items(TempContentList, HeadItem, TailItems,
-            TempsInUse0, ChosenStackVars)
+            TempsInUse0, StackVarsPrime,
+            StackIdPrime, FirstSlotNumPrime, LastSlotNumPrime)
     ->
-        StackVars = ChosenStackVars
+        StackVars = StackVarsPrime,
+        StackId = StackIdPrime,
+        FirstSlotNum = FirstSlotNumPrime,
+        LastSlotNum = LastSlotNumPrime
     ;
-        list.map_foldl(new_temp_slot, [HeadItem | TailItems], StackVars, !CI)
+        new_temp_slots([HeadItem | TailItems], StackVars,
+            StackId, FirstSlotNum, LastSlotNum, !CI)
     ),
     (
         Persistence = persistent_temp_slot,
@@ -3895,17 +4123,50 @@
     code_info::in, code_info::out) is det.
 
 new_temp_slot(Item, StackVar, !CI) :-
-    get_var_slot_count(!.CI, VarSlots),
-    get_max_temp_slot_count(!.CI, TempSlots0),
-    TempSlots = TempSlots0 + 1,
-    Slot = VarSlots + TempSlots,
-    stack_variable(!.CI, Slot, StackVar),
-    set_max_temp_slot_count(TempSlots, !CI),
+    get_var_slot_count(!.CI, VarSlotCount),
+    get_max_temp_slot_count(!.CI, TempSlotCount0),
+    TempSlotCount = TempSlotCount0 + 1,
+    SlotNum = VarSlotCount + TempSlotCount,
+    CodeModel = get_proc_model(!.CI),
+    StackId = code_model_to_main_stack(CodeModel),
+    StackVar = stack_slot_num_to_lval(StackId, SlotNum),
+    set_max_temp_slot_count(TempSlotCount, !CI),
 
     get_temp_content_map(!.CI, TempContentMap0),
     map.det_insert(TempContentMap0, StackVar, Item, TempContentMap),
     set_temp_content_map(TempContentMap, !CI).
 
+:- pred new_temp_slots(list(slot_contents)::in, list(lval)::out,
+    main_stack::out, int::out, int::out, code_info::in, code_info::out) is det.
+
+new_temp_slots(Items, StackVars, StackId, FirstSlotNum, LastSlotNum, !CI) :-
+    get_var_slot_count(!.CI, VarSlotCount),
+    get_max_temp_slot_count(!.CI, TempSlotCount0),
+    FirstSlotNum = VarSlotCount + TempSlotCount0 + 1,
+    CodeModel = get_proc_model(!.CI),
+    StackId = code_model_to_main_stack(CodeModel),
+    get_temp_content_map(!.CI, TempContentMap0),
+    record_new_temp_slots(Items, StackId, FirstSlotNum, LastSlotNum,
+        TempSlotCount0, TempSlotCount, TempContentMap0, TempContentMap,
+        StackVars),
+    set_max_temp_slot_count(TempSlotCount, !CI),
+    set_temp_content_map(TempContentMap, !CI).
+
+:- pred record_new_temp_slots(list(slot_contents)::in,
+    main_stack::in, int::in, int::out, int::in, int::out,
+    map(lval, slot_contents)::in, map(lval, slot_contents)::out,
+    list(lval)::out) is det.
+
+record_new_temp_slots([], _, !CurSlotNum, !TempSlotCount, !TempContentMap, []).
+record_new_temp_slots([Item | Items], StackId, !CurSlotNum,
+        !TempSlotCount, !TempContentMap, [StackVar | StackVars]) :-
+    StackVar = stack_slot_num_to_lval(StackId, !.CurSlotNum),
+    map.det_insert(!.TempContentMap, StackVar, Item, !:TempContentMap),
+    !:CurSlotNum = !.CurSlotNum + 1,
+    !:TempSlotCount = !.TempSlotCount + 1,
+    record_new_temp_slots(Items, StackId, !CurSlotNum,
+        !TempSlotCount, !TempContentMap, StackVars).
+
 :- pred find_unused_slot_for_item(assoc_list(lval, slot_contents)::in,
     slot_contents::in, set(lval)::in, lval::out,
     assoc_list(lval, slot_contents)::out) is semidet.
@@ -3925,47 +4186,52 @@
     ).
 
 :- pred find_unused_slots_for_items(assoc_list(lval, slot_contents)::in,
-    slot_contents::in, list(slot_contents)::in, set(lval)::in, list(lval)::out)
-    is semidet.
+    slot_contents::in, list(slot_contents)::in, set(lval)::in, list(lval)::out,
+    main_stack::out, int::out, int::out) is semidet.
 
-find_unused_slots_for_items([Head | Tail], HeadItem, TailItems,
-        TempsInUse, ChosenStackVars) :-
+find_unused_slots_for_items([Head | Tail], HeadItem, TailItems, TempsInUse,
+        ChosenStackVars, StackId, FirstSlotNum, LastSlotNum) :-
     (
         find_unused_slot_for_item([Head | Tail], HeadItem, TempsInUse,
             ChosenHeadStackVar, Remainder),
-        find_next_slots_for_items(Remainder, ChosenHeadStackVar, TailItems,
-            TempsInUse, ChosenTailStackVars)
+        ( ChosenHeadStackVar =  stackvar(N) ->
+            StackId0 = det_stack,
+            FirstSlotNum0 = N
+        ; ChosenHeadStackVar =  framevar(N) ->
+            StackId0 = nondet_stack,
+            FirstSlotNum0 = N
+        ;
+            unexpected(this_file,
+                "find_unused_slots_for_items: not stackvar or framevar")
+        ),
+        StackId1 = StackId0,
+        FirstSlotNum1 = FirstSlotNum0,
+        find_next_slots_for_items(Remainder, TailItems, TempsInUse,
+            ChosenTailStackVars, StackId1, FirstSlotNum1, LastSlotNum1)
     ->
-        ChosenStackVars = [ChosenHeadStackVar | ChosenTailStackVars]
+        ChosenStackVars = [ChosenHeadStackVar | ChosenTailStackVars],
+        StackId = StackId1,
+        FirstSlotNum = FirstSlotNum1,
+        LastSlotNum = LastSlotNum1
     ;
-        find_unused_slots_for_items(Tail, HeadItem, TailItems,
-            TempsInUse, ChosenStackVars)
+        find_unused_slots_for_items(Tail, HeadItem, TailItems, TempsInUse,
+            ChosenStackVars, StackId, FirstSlotNum, LastSlotNum)
     ).
 
 :- pred find_next_slots_for_items(assoc_list(lval, slot_contents)::in,
-    lval::in, list(slot_contents)::in, set(lval)::in, list(lval)::out)
-    is semidet.
+    list(slot_contents)::in, set(lval)::in, list(lval)::out,
+    main_stack::in, int::in, int::out) is semidet.
 
-find_next_slots_for_items([], _, [], _, []).
-find_next_slots_for_items([Head | Tail], PrevStackVar, [HeadItem | TailItems],
-        TempsInUse, [HeadStackVar | TailStackVars]) :-
+find_next_slots_for_items([], [], _, [], _, !SlotNum).
+find_next_slots_for_items([Head | Tail], [HeadItem | TailItems], TempsInUse,
+        [HeadStackVar | TailStackVars], StackId, !SlotNum) :-
     Head = HeadStackVar - HeadSlotType,
-    HeadStackVar = get_next_stack_var(PrevStackVar),
+    !:SlotNum = !.SlotNum + 1,
+    HeadStackVar = stack_slot_num_to_lval(StackId, !.SlotNum),
     HeadSlotType = HeadItem,
     \+ set.member(HeadStackVar, TempsInUse),
-    find_next_slots_for_items(Tail, HeadStackVar, TailItems,
-        TempsInUse, TailStackVars).
-
-:- func get_next_stack_var(lval) = lval.
-
-get_next_stack_var(Lval) = NextLval :-
-    ( Lval = stackvar(N) ->
-        NextLval = stackvar(N+1)
-    ; Lval = framevar(N) ->
-        NextLval = framevar(N+1)
-    ;
-        unexpected(this_file, "get_next_stack_var: not stackvar or framevar")
-    ).
+    find_next_slots_for_items(Tail, TailItems, TempsInUse,
+        TailStackVars, StackId, !SlotNum).
 
 release_temp_slot(StackVar, Persistence, !CI) :-
     get_temps_in_use(!.CI, TempsInUse0),
@@ -4033,24 +4299,6 @@
     ),
     max_var_slot_2(Ls, !Max).
 
-:- pred stack_variable(code_info::in, int::in, lval::out) is det.
-
-stack_variable(CI, Num, Lval) :-
-    ( get_proc_model(CI) = model_non ->
-        Lval = framevar(Num)
-    ;
-        Lval = stackvar(Num)
-    ).
-
-:- pred stack_variable_reference(code_info::in, int::in, rval::out) is det.
-
-stack_variable_reference(CI, Num, mem_addr(Ref)) :-
-    ( get_proc_model(CI) = model_non ->
-        Ref = framevar_ref(const(llconst_int(Num)))
-    ;
-        Ref = stackvar_ref(const(llconst_int(Num)))
-    ).
-
 %---------------------------------------------------------------------------%
 
 :- func this_file = string.
Index: compiler/commit_gen.m
===================================================================
RCS file: /home/mercury/mercury1/repository/mercury/compiler/commit_gen.m,v
retrieving revision 1.16
diff -u -b -r1.16 commit_gen.m
--- compiler/commit_gen.m	6 Jan 2007 09:23:26 -0000	1.16
+++ compiler/commit_gen.m	23 Jul 2007 15:15:47 -0000
@@ -20,11 +20,15 @@
 :- import_module hlds.hlds_goal.
 :- import_module ll_backend.code_info.
 :- import_module ll_backend.llds.
+:- import_module parse_tree.prog_data.
+
+:- import_module set.
 
 %---------------------------------------------------------------------------%
 
-:- pred generate_scope(scope_reason::in, add_trail_ops::in, code_model::in,
-    hlds_goal::in, code_tree::out, code_info::in, code_info::out) is det.
+:- pred generate_scope(scope_reason::in, code_model::in, hlds_goal_info::in,
+    set(prog_var)::in, hlds_goal::in, code_tree::out,
+    code_info::in, code_info::out) is det.
 
 %---------------------------------------------------------------------------%
 %---------------------------------------------------------------------------%
@@ -42,22 +46,28 @@
 
 %---------------------------------------------------------------------------%
 
-generate_scope(Reason, AddTrailOps, OuterCodeModel, Goal, Code, !CI) :-
+generate_scope(Reason, OuterCodeModel, OuterGoalInfo,
+        ForwardLiveVarsBeforeGoal, Goal, Code, !CI) :-
     (
         Reason = trace_goal(_, MaybeTraceRuntimeCond, _, _, _),
         MaybeTraceRuntimeCond = yes(_)
     ->
         % These goals should have been transformed into other forms of goals
-        % by simplify.m at the end of semantics analysis.
+        % by simplify.m at the end of semantic analysis.
         unexpected(this_file, "generate_scope: trace_goal")
     ;
-        generate_commit(AddTrailOps, OuterCodeModel, Goal, Code, !CI)
+        generate_commit(OuterCodeModel, OuterGoalInfo,
+            ForwardLiveVarsBeforeGoal, Goal, Code, !CI)
     ).
 
-:- pred generate_commit(add_trail_ops::in, code_model::in,
+:- pred generate_commit(code_model::in, hlds_goal_info::in, set(prog_var)::in,
     hlds_goal::in, code_tree::out, code_info::in, code_info::out) is det.
 
-generate_commit(AddTrailOps, OuterCodeModel, Goal, Code, !CI) :-
+generate_commit(OuterCodeModel, OuterGoalInfo, ForwardLiveVarsBeforeGoal,
+        Goal, Code, !CI) :-
+    AddTrailOps = should_add_trail_ops(!.CI, OuterGoalInfo),
+    AddRegionOps = should_add_region_ops(!.CI, OuterGoalInfo),
+
     Goal = hlds_goal(_, InnerGoalInfo),
     goal_info_get_code_model(InnerGoalInfo, InnerCodeModel),
     (
@@ -67,15 +77,15 @@
             code_gen.generate_goal(InnerCodeModel, Goal, Code, !CI)
         ;
             InnerCodeModel = model_semi,
-            unexpected(this_file, "generate_commit: " ++
-                "semidet model in det context")
+            unexpected(this_file,
+                "generate_commit: semidet model in det context")
         ;
             InnerCodeModel = model_non,
-            code_info.prepare_for_det_commit(AddTrailOps, CommitInfo,
-                PreCommit, !CI),
+            code_info.prepare_for_det_commit(AddTrailOps, AddRegionOps,
+                ForwardLiveVarsBeforeGoal, CommitInfo, PreCommit, !CI),
             code_gen.generate_goal(InnerCodeModel, Goal, GoalCode, !CI),
             code_info.generate_det_commit(CommitInfo, Commit, !CI),
-            Code = tree(PreCommit, tree(GoalCode, Commit))
+            Code = tree_list([PreCommit, GoalCode, Commit])
         )
     ;
         OuterCodeModel = model_semi,
@@ -87,11 +97,11 @@
             code_gen.generate_goal(InnerCodeModel, Goal, Code, !CI)
         ;
             InnerCodeModel = model_non,
-            code_info.prepare_for_semi_commit(AddTrailOps, CommitInfo,
-                PreCommit, !CI),
+            code_info.prepare_for_semi_commit(AddTrailOps, AddRegionOps,
+                ForwardLiveVarsBeforeGoal, CommitInfo, PreCommit, !CI),
             code_gen.generate_goal(InnerCodeModel, Goal, GoalCode, !CI),
             code_info.generate_semi_commit(CommitInfo, Commit, !CI),
-            Code = tree(PreCommit, tree(GoalCode, Commit))
+            Code = tree_list([PreCommit, GoalCode, Commit])
         )
     ;
         OuterCodeModel = model_non,
Index: compiler/continuation_info.m
===================================================================
RCS file: /home/mercury/mercury1/repository/mercury/compiler/continuation_info.m,v
retrieving revision 1.88
diff -u -b -r1.88 continuation_info.m
--- compiler/continuation_info.m	6 Jan 2007 10:56:11 -0000	1.88
+++ compiler/continuation_info.m	30 Jul 2007 04:35:12 -0000
@@ -312,16 +312,19 @@
             ).
 
 :- type slot_contents
-    --->    ticket          % A ticket (trail pointer).
-    ;       ticket_counter  % A copy of the ticket counter.
-    ;       trace_data
-    ;       lookup_disj_cur
-    ;       lookup_switch_cur
-    ;       lookup_switch_max
-    ;       sync_term       % A syncronization term used
+    --->    slot_ticket             % A ticket (trail pointer).
+    ;       slot_ticket_counter     % A copy of the ticket counter.
+    ;       slot_trace_data
+    ;       slot_lookup_disj_cur
+    ;       slot_lookup_switch_cur
+    ;       slot_lookup_switch_max
+    ;       slot_sync_term          % A syncronization term used
                             % at the end of par_conjs.
                             % See par_conj_gen.m for details.
-    ;       lval(lval).
+    ;       slot_region_ite
+    ;       slot_region_disj
+    ;       slot_region_commit
+    ;       slot_lval(lval).
 
     % Call maybe_process_proc_llds on the code of every procedure in the list.
     %
@@ -883,35 +886,38 @@
 
 :- pred live_value_type(slot_contents::in, live_value_type::out) is det.
 
-live_value_type(lval(succip), live_value_succip).
-live_value_type(lval(hp), live_value_hp).
-live_value_type(lval(maxfr), live_value_maxfr).
-live_value_type(lval(curfr), live_value_curfr).
-live_value_type(lval(succfr_slot(_)), live_value_unwanted).
-live_value_type(lval(prevfr_slot(_)), live_value_unwanted).
-live_value_type(lval(redofr_slot(_)), live_value_unwanted).
-live_value_type(lval(redoip_slot(_)), live_value_unwanted).
-live_value_type(lval(succip_slot(_)), live_value_unwanted).
-live_value_type(lval(sp), live_value_unwanted).
-live_value_type(lval(parent_sp), live_value_unwanted).
-live_value_type(lval(lvar(_)), live_value_unwanted).
-live_value_type(lval(field(_, _, _)), live_value_unwanted).
-live_value_type(lval(temp(_, _)), live_value_unwanted).
-live_value_type(lval(reg(_, _)), live_value_unwanted).
-live_value_type(lval(stackvar(_)), live_value_unwanted).
-live_value_type(lval(parent_stackvar(_)), live_value_unwanted).
-live_value_type(lval(framevar(_)), live_value_unwanted).
-live_value_type(lval(mem_ref(_)), live_value_unwanted). % XXX
-live_value_type(lval(global_var_ref(_)), live_value_unwanted).
-live_value_type(ticket, live_value_unwanted).
+live_value_type(slot_lval(succip), live_value_succip).
+live_value_type(slot_lval(hp), live_value_hp).
+live_value_type(slot_lval(maxfr), live_value_maxfr).
+live_value_type(slot_lval(curfr), live_value_curfr).
+live_value_type(slot_lval(succfr_slot(_)), live_value_unwanted).
+live_value_type(slot_lval(prevfr_slot(_)), live_value_unwanted).
+live_value_type(slot_lval(redofr_slot(_)), live_value_unwanted).
+live_value_type(slot_lval(redoip_slot(_)), live_value_unwanted).
+live_value_type(slot_lval(succip_slot(_)), live_value_unwanted).
+live_value_type(slot_lval(sp), live_value_unwanted).
+live_value_type(slot_lval(parent_sp), live_value_unwanted).
+live_value_type(slot_lval(lvar(_)), live_value_unwanted).
+live_value_type(slot_lval(field(_, _, _)), live_value_unwanted).
+live_value_type(slot_lval(temp(_, _)), live_value_unwanted).
+live_value_type(slot_lval(reg(_, _)), live_value_unwanted).
+live_value_type(slot_lval(stackvar(_)), live_value_unwanted).
+live_value_type(slot_lval(parent_stackvar(_)), live_value_unwanted).
+live_value_type(slot_lval(framevar(_)), live_value_unwanted).
+live_value_type(slot_lval(mem_ref(_)), live_value_unwanted). % XXX
+live_value_type(slot_lval(global_var_ref(_)), live_value_unwanted).
+live_value_type(slot_ticket, live_value_unwanted).
     % XXX we may need to modify this, if the GC is going to garbage-collect
     % the trail.
-live_value_type(ticket_counter, live_value_unwanted).
-live_value_type(lookup_disj_cur, live_value_unwanted).
-live_value_type(lookup_switch_cur, live_value_unwanted).
-live_value_type(lookup_switch_max, live_value_unwanted).
-live_value_type(sync_term, live_value_unwanted).
-live_value_type(trace_data, live_value_unwanted).
+live_value_type(slot_ticket_counter, live_value_unwanted).
+live_value_type(slot_lookup_disj_cur, live_value_unwanted).
+live_value_type(slot_lookup_switch_cur, live_value_unwanted).
+live_value_type(slot_lookup_switch_max, live_value_unwanted).
+live_value_type(slot_sync_term, live_value_unwanted).
+live_value_type(slot_trace_data, live_value_unwanted).
+live_value_type(slot_region_ite, live_value_region_ite).
+live_value_type(slot_region_disj, live_value_region_disj).
+live_value_type(slot_region_commit, live_value_region_commit).
 
 %-----------------------------------------------------------------------------%
 
Index: compiler/disj_gen.m
===================================================================
RCS file: /home/mercury/mercury1/repository/mercury/compiler/disj_gen.m,v
retrieving revision 1.102
diff -u -b -r1.102 disj_gen.m
--- compiler/disj_gen.m	13 Jul 2007 03:27:26 -0000	1.102
+++ compiler/disj_gen.m	30 Jul 2007 04:32:38 -0000
@@ -25,8 +25,8 @@
 
 %-----------------------------------------------------------------------------%
 
-:- pred generate_disj(add_trail_ops::in, code_model::in, list(hlds_goal)::in,
-    hlds_goal_info::in, code_tree::out, code_info::in, code_info::out) is det.
+:- pred generate_disj(code_model::in, list(hlds_goal)::in, hlds_goal_info::in,
+    code_tree::out, code_info::in, code_info::out) is det.
 
 %-----------------------------------------------------------------------------%
 %-----------------------------------------------------------------------------%
@@ -58,32 +58,40 @@
 
 %-----------------------------------------------------------------------------%
 
-generate_disj(AddTrailOps, CodeModel, Goals, DisjGoalInfo, Code, !CI) :-
+generate_disj(CodeModel, Goals, DisjGoalInfo, Code, !CI) :-
     (
         Goals = [],
-        ( CodeModel = model_semi ->
+        (
+            CodeModel = model_semi,
             code_info.generate_failure(Code, !CI)
         ;
+            ( CodeModel = model_det
+            ; CodeModel = model_non
+            ),
             unexpected(this_file, "generate_disj: empty disjunction.")
         )
     ;
         Goals = [Goal | _],
         Goal = hlds_goal(_, GoalInfo),
         goal_info_get_resume_point(GoalInfo, Resume),
-        ( Resume = resume_point(ResumeVarsPrime, _) ->
+        (
+            Resume = resume_point(ResumeVarsPrime, _),
             ResumeVars = ResumeVarsPrime
         ;
+            Resume = no_resume_point,
             set.init(ResumeVars)
         ),
+        AddTrailOps = should_add_trail_ops(!.CI, GoalInfo),
+        AddRegionOps = should_add_region_ops(!.CI, GoalInfo),
         (
             CodeModel = model_non,
-            is_lookup_disj(AddTrailOps, ResumeVars, Goals, DisjGoalInfo,
-                LookupDisjInfo, !CI)
+            is_lookup_disj(AddTrailOps, AddRegionOps, ResumeVars, Goals,
+                DisjGoalInfo, LookupDisjInfo, !CI)
         ->
             generate_lookup_disj(ResumeVars, LookupDisjInfo, Code, !CI)
         ;
-            generate_real_disj(AddTrailOps, CodeModel, ResumeVars, Goals,
-                DisjGoalInfo, Code, !CI)
+            generate_real_disj(AddTrailOps, AddRegionOps, CodeModel,
+                ResumeVars, Goals, DisjGoalInfo, Code, !CI)
         )
     ).
 
@@ -114,12 +122,12 @@
                 ldi_field_types         :: list(llds_type)
             ).
 
-:- pred is_lookup_disj(bool::in, set(prog_var)::in, list(hlds_goal)::in,
-    hlds_goal_info::in, lookup_disj_info::out, code_info::in, code_info::out)
-    is semidet.
+:- pred is_lookup_disj(add_trail_ops::in, add_region_ops::in,
+    set(prog_var)::in, list(hlds_goal)::in, hlds_goal_info::in,
+    lookup_disj_info::out, code_info::in, code_info::out) is semidet.
 
-is_lookup_disj(AddTrailOps, ResumeVars, Goals, DisjGoalInfo, LookupDisjInfo,
-        !CI) :-
+is_lookup_disj(AddTrailOps, AddRegionOps, ResumeVars, Goals, DisjGoalInfo,
+        LookupDisjInfo, !CI) :-
     code_info.get_maybe_trace_info(!.CI, MaybeTraceInfo),
     MaybeTraceInfo = no,
 
@@ -127,6 +135,10 @@
     code_info.get_globals(!.CI, Globals),
     globals.lookup_bool_option(Globals, static_ground_terms, yes),
 
+    % XXX The code to generate lookup disjunctions hasn't yet been updated
+    % to handle region operations.
+    AddRegionOps = do_not_add_region_ops,
+
     % Since we generate two code sequences, one for the first solution and one
     % for all the later solutions, we don't get any benefit over the code
     % sequence generated by generate_real_disj unless there are at least three
@@ -145,7 +157,7 @@
     % However, if we are inside an outer branched control structure
     % (disjunction, switch, if-then-else), it may be released (implicitly)
     % when we get into the next branch of that outer control structure.
-    code_info.acquire_temp_slot(lookup_disj_cur, non_persistent_temp_slot,
+    code_info.acquire_temp_slot(slot_lookup_disj_cur, non_persistent_temp_slot,
         CurSlot, !CI),
     code_info.maybe_save_ticket(AddTrailOps, SaveTicketCode, MaybeTicketSlot,
         !CI),
@@ -295,26 +307,40 @@
 
     code_info.after_all_branches(StoreMap, MaybeEnd, !CI),
 
-    EndLabelCode = node([
-        llds_instr(label(EndLabel), "end of lookup disj")
-    ]),
-
+    EndLabelCode = node([llds_instr(label(EndLabel), "end of lookup disj")]),
     Comment = node([llds_instr(comment("lookup disj"), "")]),
-    Code = tree_list([Comment, FlushCode, BaseRegInitCode,
-        SaveSlotCode, SaveTicketCode, SaveHpCode, PrepareHijackCode,
-        UpdateRedoipCode, FirstFlushResumeVarsCode, FirstBranchEndCode,
-        GotoEndCode, ResumePointCode, RestoreTicketCode, RestoreHpCode,
-        TestMoreSolnsCode, UndoHijackCode, AfterUndoLabelCode,
-        LaterFlushResumeVarsCode, LaterBranchEndCode, EndLabelCode]).
+
+    Code = tree_list([
+        Comment,
+        FlushCode,
+        BaseRegInitCode,
+        SaveSlotCode,
+        SaveTicketCode,
+        SaveHpCode,
+        PrepareHijackCode,
+        UpdateRedoipCode,
+        FirstFlushResumeVarsCode,
+        FirstBranchEndCode,
+        GotoEndCode,
+        ResumePointCode,
+        RestoreTicketCode,
+        RestoreHpCode,
+        TestMoreSolnsCode,
+        UndoHijackCode,
+        AfterUndoLabelCode,
+        LaterFlushResumeVarsCode,
+        LaterBranchEndCode,
+        EndLabelCode
+    ]).
 
 %---------------------------------------------------------------------------%
 
-:- pred generate_real_disj(bool::in, code_model::in, set(prog_var)::in,
-    list(hlds_goal)::in, hlds_goal_info::in, code_tree::out,
-    code_info::in, code_info::out) is det.
+:- pred generate_real_disj(add_trail_ops::in, add_region_ops::in,
+    code_model::in, set(prog_var)::in, list(hlds_goal)::in, hlds_goal_info::in,
+    code_tree::out, code_info::in, code_info::out) is det.
 
-generate_real_disj(AddTrailOps, CodeModel, ResumeVars, Goals, DisjGoalInfo,
-        Code, !CI)  :-
+generate_real_disj(AddTrailOps, AddRegionOps, CodeModel, ResumeVars, Goals,
+        DisjGoalInfo, Code, !CI)  :-
     % Make sure that the variables whose values will be needed on backtracking
     % to any disjunct are materialized into registers or stack slots. Their
     % locations are recorded in ResumeMap.
@@ -331,15 +357,27 @@
 
     % If we are using a grade in which we can recover memory by saving
     % and restoring the heap pointer, set up for doing so if necessary.
-    ( CodeModel = model_non ->
+    (
+        CodeModel = model_non,
         % With nondet disjunctions, we must recover memory across all
         % disjuncts, even disjuncts that cannot themselves allocate memory,
         % since we can backtrack to disjunct N after control leaves
         % disjunct N-1.
         globals.lookup_bool_option(Globals, reclaim_heap_on_nondet_failure,
             ReclaimHeap),
-        code_info.maybe_save_hp(ReclaimHeap, SaveHpCode, MaybeHpSlot, !CI)
+        code_info.maybe_save_hp(ReclaimHeap, SaveHpCode, MaybeHpSlot, !CI),
+
+        maybe_create_disj_region_frame(AddRegionOps, DisjGoalInfo,
+            FirstRegionCode, LaterRegionCode, LastRegionCode,
+            _RegionStackVars, !CI),
+        % We can't release any of the stack slots holding the embedded stack
+        % frame, since we can't let code to the right of the disjunction reuse
+        % any of those slots.
+        RegionStackVarsToRelease = []
     ;
+        ( CodeModel = model_det
+        ; CodeModel = model_semi
+        ),
         % With other disjunctions, we can backtrack to disjunct N only from
         % disjunct N-1, so if disjunct N-1 does not allocate memory, we need
         % not recover memory across it. Since it is possible (and common)
@@ -348,7 +386,22 @@
         globals.lookup_bool_option(Globals, reclaim_heap_on_semidet_failure,
             ReclaimHeap),
         SaveHpCode = empty,
-        MaybeHpSlot = no
+        MaybeHpSlot = no,
+
+        % XXX The condition should succeed only if some disjunct performs
+        % some region operations (allocation or removal). The HLDS does not yet
+        % contain the information we need to decide whether this is the case.
+        ( semidet_succeed ->
+            maybe_create_disj_region_frame(AddRegionOps, DisjGoalInfo,
+                FirstRegionCode, LaterRegionCode, LastRegionCode,
+                RegionStackVars, !CI),
+            RegionStackVarsToRelease = RegionStackVars
+        ;
+            FirstRegionCode = empty,
+            LaterRegionCode = empty,
+            LastRegionCode = empty,
+            RegionStackVarsToRelease = []
+        )
     ),
 
     % Save the values of any stack slots we may hijack, and if necessary,
@@ -361,33 +414,45 @@
     code_info.remember_position(!.CI, BranchStart),
     generate_disjuncts(Goals, CodeModel, ResumeMap, no, HijackInfo,
         DisjGoalInfo, EndLabel, ReclaimHeap, MaybeHpSlot, MaybeTicketSlot,
-        BranchStart, no, MaybeEnd, GoalsCode, !CI),
+        LaterRegionCode, LastRegionCode, BranchStart, no, MaybeEnd, GoalsCode,
+        !CI),
 
     goal_info_get_store_map(DisjGoalInfo, StoreMap),
     code_info.after_all_branches(StoreMap, MaybeEnd, !CI),
-    ( CodeModel = model_non ->
+    code_info.release_several_temp_slots(RegionStackVarsToRelease,
+        non_persistent_temp_slot, !CI),
+    (
+        CodeModel = model_non,
         code_info.set_resume_point_to_unknown(!CI)
     ;
-        true
+        ( CodeModel = model_det
+        ; CodeModel = model_semi
+        )
+        % The resume point is unchanged.
     ),
-    Code = tree_list([FlushCode, SaveTicketCode, SaveHpCode, PrepareHijackCode,
-         GoalsCode]).
 
-%---------------------------------------------------------------------------%
+    Code = tree_list([
+        FlushCode,
+        SaveTicketCode,
+        SaveHpCode,
+        FirstRegionCode,
+        PrepareHijackCode,
+        GoalsCode
+    ]).
 
 :- pred generate_disjuncts(list(hlds_goal)::in, code_model::in,
     resume_map::in, maybe(resume_point_info)::in, disj_hijack_info::in,
     hlds_goal_info::in, label::in, bool::in, maybe(lval)::in,
-    maybe(lval)::in, position_info::in, maybe(branch_end_info)::in,
-    maybe(branch_end_info)::out, code_tree::out, code_info::in,
-    code_info::out) is det.
+    maybe(lval)::in, code_tree::in, code_tree::in,
+    position_info::in, maybe(branch_end_info)::in, maybe(branch_end_info)::out,
+    code_tree::out, code_info::in, code_info::out) is det.
 
-generate_disjuncts([], _, _, _, _, _, _, _, _, _, _, _, _, _, !CI) :-
+generate_disjuncts([], _, _, _, _, _, _, _, _, _, _, _, _, _, _, _, !CI) :-
     unexpected(this_file, "generate_disjuncts: empty disjunction!").
 generate_disjuncts([Goal0 | Goals], CodeModel, FullResumeMap,
         MaybeEntryResumePoint, HijackInfo, DisjGoalInfo, EndLabel, ReclaimHeap,
-        MaybeHpSlot0, MaybeTicketSlot, BranchStart0, MaybeEnd0, MaybeEnd,
-        Code, !CI) :-
+        MaybeHpSlot0, MaybeTicketSlot, LaterRegionCode, LastRegionCode,
+        BranchStart0, MaybeEnd0, MaybeEnd, Code, !CI) :-
 
     code_info.reset_to_position(BranchStart0, !CI),
 
@@ -404,7 +469,8 @@
 
     Goal0 = hlds_goal(GoalExpr0, GoalInfo0),
     goal_info_get_resume_point(GoalInfo0, Resume),
-    ( Resume = resume_point(ResumeVars, ResumeLocs) ->
+    (
+        Resume = resume_point(ResumeVars, ResumeLocs),
         % Emit code for a non-last disjunct, including setting things
         % up for the execution of the next disjunct.
 
@@ -467,7 +533,9 @@
         goal_info_get_code_model(GoalInfo, GoalCodeModel),
         code_gen.generate_goal(GoalCodeModel, Goal, GoalCode, !CI),
 
-        ( CodeModel = model_non ->
+        (
+            CodeModel = model_non,
+
             % We can backtrack to the next disjunct from outside, so we make
             % sure every variable in the resume set is in its stack slot.
             code_info.flush_resume_vars_to_stack(ResumeVarsCode, !CI),
@@ -477,6 +545,10 @@
             % after the disjunction.
             PruneTicketCode = empty
         ;
+            ( CodeModel = model_det
+            ; CodeModel = model_semi
+            ),
+
             ResumeVarsCode = empty,
 
             code_info.maybe_release_hp(MaybeHpSlot, !CI),
@@ -509,14 +581,28 @@
 
         generate_disjuncts(Goals, CodeModel, FullResumeMap,
             yes(NextResumePoint), HijackInfo, DisjGoalInfo,
-            EndLabel, ReclaimHeap, MaybeHpSlot, MaybeTicketSlot, BranchStart,
+            EndLabel, ReclaimHeap, MaybeHpSlot, MaybeTicketSlot,
+            LaterRegionCode, LastRegionCode, BranchStart,
             MaybeEnd1, MaybeEnd, RestCode, !CI),
 
-        Code = tree_list([EntryResumePointCode, RestoreHpCode,
-            RestoreTicketCode, SaveHpCode, ModContCode, TraceCode,
-            GoalCode, ResumeVarsCode, PruneTicketCode, SaveCode,
-            BranchCode, RestCode])
+        Code = tree_list([
+            EntryResumePointCode,
+            RestoreHpCode,
+            RestoreTicketCode,
+            SaveHpCode,
+            LaterRegionCode,
+            ModContCode,
+            TraceCode,
+            GoalCode,
+            ResumeVarsCode,
+            PruneTicketCode,
+            SaveCode,
+            BranchCode,
+            RestCode
+        ])
     ;
+        Resume = no_resume_point,
+
         % Emit code for the last disjunct.
 
         % Restore the heap pointer and solver state if necessary.
@@ -537,12 +623,169 @@
         EndCode = node([
             llds_instr(label(EndLabel), "End of nondet disj")
         ]),
-        Code = tree_list([EntryResumePointCode, TraceCode, RestoreHpCode,
-            RestoreTicketCode, UndoCode, GoalCode, SaveCode, EndCode])
+        Code = tree_list([
+            EntryResumePointCode,
+            TraceCode,      % XXX Should this be after LastRegionCode?
+            RestoreHpCode,
+            RestoreTicketCode,
+            LastRegionCode,
+            UndoCode,
+            GoalCode,
+            SaveCode,
+            EndCode
+        ])
     ).
 
 %-----------------------------------------------------------------------------%
 
+:- pred maybe_create_disj_region_frame(add_region_ops::in, hlds_goal_info::in,
+    code_tree::out, code_tree::out, code_tree::out, list(lval)::out,
+    code_info::in, code_info::out) is det.
+
+maybe_create_disj_region_frame(DisjRegionOps, _DisjGoalInfo,
+        FirstCode, LaterCode, LastCode, StackVars, !CI) :-
+    (
+        DisjRegionOps = do_not_add_region_ops,
+        FirstCode = empty,
+        LaterCode = empty,
+        LastCode = empty,
+        StackVars = []
+    ;
+        DisjRegionOps = add_region_ops,
+        code_info.get_forward_live_vars(!.CI, ForwardLiveVars),
+        LiveRegionVars = filter_region_vars(!.CI, ForwardLiveVars),
+
+        % XXX In computing both ProtectRegionVars and SnapshotRegionVars,
+        % we should intersect LiveRegionVars with the set of region variables
+        % whose regions (the regions themselves, not their variables) are live
+        % at the starts of some later disjuncts (i.e. aren't used only in the
+        % first disjunct). We don't yet gather this information.
+        %
+        % XXX In computing ProtectRegionVars, we should also delete any
+        % variables that are statically known to be already protected by
+        % an outer disjunction, but we don't yet have the program analysis
+        % required to gather such information.
+        ProtectRegionVars = LiveRegionVars,
+        SnapshotRegionVars = LiveRegionVars,
+
+        ProtectRegionVarList = set.to_sorted_list(ProtectRegionVars),
+        SnapshotRegionVarList = set.to_sorted_list(SnapshotRegionVars),
+
+        list.length(ProtectRegionVarList, NumProtectRegionVars),
+        list.length(SnapshotRegionVarList, NumSnapshotRegionVars),
+
+        code_info.get_globals(!.CI, Globals),
+        globals.lookup_int_option(Globals, size_region_disj_fixed,
+            FixedSize),
+        globals.lookup_int_option(Globals, size_region_disj_protect,
+            ProtectSize),
+        globals.lookup_int_option(Globals, size_region_disj_snapshot,
+            SnapshotSize),
+        FrameSize = FixedSize
+            + ProtectSize * NumProtectRegionVars
+            + SnapshotSize * NumSnapshotRegionVars,
+
+        Items = list.duplicate(FrameSize, slot_region_disj),
+        acquire_several_temp_slots(Items, non_persistent_temp_slot,
+            StackVars, MainStackId, FirstSlotNum, LastSlotNum, !CI),
+        EmbeddedStackFrame = embedded_stack_frame_id(MainStackId,
+            FirstSlotNum, LastSlotNum),
+        FirstNonFixedAddr =
+            first_nonfixed_embedded_slot_addr(EmbeddedStackFrame, FixedSize),
+        acquire_reg(reg_r, ProtectNumRegLval, !CI),
+        acquire_reg(reg_r, SnapshotNumRegLval, !CI),
+        acquire_reg(reg_r, AddrRegLval, !CI),
+        PushInitCode = node([
+            llds_instr(
+                push_region_frame(region_stack_disj, EmbeddedStackFrame),
+                "Save stack pointer of embedded region nondet stack"),
+            llds_instr(
+                assign(ProtectNumRegLval, const(llconst_int(0))),
+                "Initialize number of protect_infos"),
+            llds_instr(
+                assign(SnapshotNumRegLval, const(llconst_int(0))),
+                "Initialize number of snapshot_infos"),
+            llds_instr(
+                assign(AddrRegLval, FirstNonFixedAddr),
+                "Initialize pointer to nonfixed part of embedded frame")
+        ]),
+        disj_protect_regions(ProtectNumRegLval, AddrRegLval,
+            EmbeddedStackFrame, ProtectRegionVarList, ProtectRegionCode,
+            !CI),
+        disj_alloc_snapshot_regions(SnapshotNumRegLval, AddrRegLval,
+            EmbeddedStackFrame, SnapshotRegionVarList, SnapshotRegionCode,
+            !CI),
+        SetCode = node([
+            llds_instr(
+                region_set_fixed_slot(region_set_disj_num_protects,
+                    EmbeddedStackFrame, lval(ProtectNumRegLval)),
+                "Store the number of protect_infos"),
+            llds_instr(
+                region_set_fixed_slot(region_set_disj_num_snapshots,
+                    EmbeddedStackFrame, lval(SnapshotNumRegLval)),
+                "Store the number of snapshot_infos")
+        ]),
+        release_reg(ProtectNumRegLval, !CI),
+        release_reg(SnapshotNumRegLval, !CI),
+        release_reg(AddrRegLval, !CI),
+
+        FirstCode = tree_list([
+            PushInitCode,
+            ProtectRegionCode,
+            SnapshotRegionCode,
+            SetCode
+        ]),
+        LaterCode = node([
+            llds_instr(
+                use_and_maybe_pop_region_frame(region_disj_later,
+                    EmbeddedStackFrame),
+                "region enter later disjunct")
+        ]),
+        LastCode = node([
+            llds_instr(
+                use_and_maybe_pop_region_frame(region_disj_last,
+                    EmbeddedStackFrame),
+                "region enter last disjunct")
+        ])
+    ).
+
+:- pred disj_protect_regions(lval::in, lval::in, embedded_stack_frame_id::in,
+    list(prog_var)::in, code_tree::out, code_info::in, code_info::out) is det.
+
+disj_protect_regions(_, _, _, [], empty, !CI).
+disj_protect_regions(NumLval, AddrLval, EmbeddedStackFrame,
+        [RegionVar | RegionVars], tree(Code, Codes), !CI) :-
+    produce_variable(RegionVar, ProduceVarCode, RegionVarRval, !CI),
+    SaveCode = node([
+        llds_instr(
+            region_fill_frame(region_fill_disj_protect,
+                EmbeddedStackFrame, RegionVarRval, NumLval, AddrLval),
+            "disj protect the region if needed")
+    ]),
+    Code = tree(ProduceVarCode, SaveCode),
+    disj_protect_regions(NumLval, AddrLval, EmbeddedStackFrame,
+        RegionVars, Codes, !CI).
+
+:- pred disj_alloc_snapshot_regions(lval::in, lval::in,
+    embedded_stack_frame_id::in, list(prog_var)::in, code_tree::out,
+    code_info::in, code_info::out) is det.
+
+disj_alloc_snapshot_regions(_, _, _, [], empty, !CI).
+disj_alloc_snapshot_regions(NumLval, AddrLval, EmbeddedStackFrame,
+        [RegionVar | RegionVars], tree(Code, Codes), !CI) :-
+    produce_variable(RegionVar, ProduceVarCode, RegionVarRval, !CI),
+    SaveCode = node([
+        llds_instr(
+            region_fill_frame(region_fill_disj_snapshot,
+                EmbeddedStackFrame, RegionVarRval, NumLval, AddrLval),
+            "take alloc snapshot of the region")
+    ]),
+    Code = tree(ProduceVarCode, SaveCode),
+    disj_alloc_snapshot_regions(NumLval, AddrLval, EmbeddedStackFrame,
+        RegionVars, Codes, !CI).
+
+%-----------------------------------------------------------------------------%
+
 :- func this_file = string.
 
 this_file = "disj_gen.m".
Index: compiler/dupelim.m
===================================================================
RCS file: /home/mercury/mercury1/repository/mercury/compiler/dupelim.m,v
retrieving revision 1.90
diff -u -b -r1.90 dupelim.m
--- compiler/dupelim.m	9 Jul 2007 13:28:30 -0000	1.90
+++ compiler/dupelim.m	30 Jul 2007 04:49:43 -0000
@@ -108,9 +108,9 @@
         process_clusters(Clusters, LabelSeq0, LabelSeq, BlockMap0, BlockMap,
             ReplMap0, ReplMap),
         flatten_basic_blocks(LabelSeq, BlockMap, Instrs1),
-        opt_util.replace_labels_instruction_list(Instrs1,
-            ReplMap, yes, no, Instrs2),
-        list.append(Comments, Instrs2, Instrs)
+        opt_util.replace_labels_instruction_list(Instrs1, Instrs2,
+            ReplMap, yes, no),
+        Instrs = Comments ++ Instrs2
     ).
 
 %-----------------------------------------------------------------------------%
@@ -319,123 +319,113 @@
     %
 :- pred standardize_instr(instr::in, instr::out) is det.
 
-standardize_instr(Instr1, Instr) :-
+standardize_instr(Instr0, Instr) :-
     (
-        Instr1 = comment(_),
-        Instr = Instr1
-    ;
-        Instr1 = livevals(_),
-        Instr = Instr1
-    ;
-        Instr1 = block(_, _, _),
-        Instr = Instr1
-    ;
-        Instr1 = assign(Lval1, Rval1),
-        standardize_lval(Lval1, Lval),
-        standardize_rval(Rval1, Rval),
+        Instr0 = assign(Lval0, Rval0),
+        standardize_lval(Lval0, Lval),
+        standardize_rval(Rval0, Rval),
         Instr = assign(Lval, Rval)
     ;
-        Instr1 = llcall(_, _, _, _, _, _),
-        Instr = Instr1
-    ;
-        Instr1 = mkframe(_, _),
-        Instr = Instr1
-    ;
-        Instr1 = label(_),
-        Instr = Instr1
-    ;
-        Instr1 = goto(_),
-        Instr = Instr1
-    ;
-        Instr1 = computed_goto(_, _),
-        Instr = Instr1
+        Instr0 = keep_assign(Lval0, Rval0),
+        standardize_lval(Lval0, Lval),
+        standardize_rval(Rval0, Rval),
+        Instr = keep_assign(Lval, Rval)
     ;
-        Instr1 = arbitrary_c_code(_, _, _),
-        Instr = Instr1
-    ;
-        Instr1 = if_val(Rval1, CodeAddr),
-        standardize_rval(Rval1, Rval),
+        Instr0 = if_val(Rval0, CodeAddr),
+        standardize_rval(Rval0, Rval),
         Instr = if_val(Rval, CodeAddr)
     ;
-        Instr1 = save_maxfr(Lval1),
-        standardize_lval(Lval1, Lval),
+        Instr0 = save_maxfr(Lval0),
+        standardize_lval(Lval0, Lval),
         Instr = save_maxfr(Lval)
     ;
-        Instr1 = restore_maxfr(Lval1),
-        standardize_lval(Lval1, Lval),
+        Instr0 = restore_maxfr(Lval0),
+        standardize_lval(Lval0, Lval),
         Instr = restore_maxfr(Lval)
     ;
-        Instr1 = incr_hp(Lval1, MaybeTag, MaybeOffset, Rval1, Msg,
-            MayUseAtomic, MaybeRegionRval1),
-        standardize_lval(Lval1, Lval),
-        standardize_rval(Rval1, Rval),
+        Instr0 = incr_hp(Lval0, MaybeTag, MaybeOffset, Rval0, Msg,
+            MayUseAtomic, MaybeRegionRval0),
+        standardize_lval(Lval0, Lval),
+        standardize_rval(Rval0, Rval),
         (
-            MaybeRegionRval1 = yes(RegionRval1),
-            standardize_rval(RegionRval1, RegionRval),
+            MaybeRegionRval0 = yes(RegionRval0),
+            standardize_rval(RegionRval0, RegionRval),
             MaybeRegionRval = yes(RegionRval)
         ;
-            MaybeRegionRval1 = no,
-            MaybeRegionRval = MaybeRegionRval1
+            MaybeRegionRval0 = no,
+            MaybeRegionRval = MaybeRegionRval0
         ),
         Instr = incr_hp(Lval, MaybeTag, MaybeOffset, Rval, Msg,
             MayUseAtomic, MaybeRegionRval)
     ;
-        Instr1 = mark_hp(Lval1),
-        standardize_lval(Lval1, Lval),
+        Instr0 = mark_hp(Lval0),
+        standardize_lval(Lval0, Lval),
         Instr = mark_hp(Lval)
     ;
-        Instr1 = restore_hp(Rval1),
-        standardize_rval(Rval1, Rval),
+        Instr0 = restore_hp(Rval0),
+        standardize_rval(Rval0, Rval),
         Instr = restore_hp(Rval)
     ;
-        Instr1 = free_heap(Rval1),
-        standardize_rval(Rval1, Rval),
+        Instr0 = region_fill_frame(FillOp, EmbeddedStackFrame, IdRval0,
+            NumLval0, AddrLval0),
+        standardize_rval(IdRval0, IdRval),
+        standardize_lval(NumLval0, NumLval),
+        standardize_lval(AddrLval0, AddrLval),
+        Instr = region_fill_frame(FillOp, EmbeddedStackFrame, IdRval,
+            NumLval, AddrLval)
+    ;
+        Instr0 = region_set_fixed_slot(SetOp, EmbeddedStackFrame, ValueRval0),
+        standardize_rval(ValueRval0, ValueRval),
+        Instr = region_set_fixed_slot(SetOp, EmbeddedStackFrame, ValueRval)
+    ;
+        Instr0 = free_heap(Rval0),
+        standardize_rval(Rval0, Rval),
         Instr = free_heap(Rval)
     ;
-        Instr1 = store_ticket(Lval1),
-        standardize_lval(Lval1, Lval),
+        Instr0 = store_ticket(Lval0),
+        standardize_lval(Lval0, Lval),
         Instr = store_ticket(Lval)
     ;
-        Instr1 = reset_ticket(Rval1, Reason),
-        standardize_rval(Rval1, Rval),
+        Instr0 = reset_ticket(Rval0, Reason),
+        standardize_rval(Rval0, Rval),
         Instr = reset_ticket(Rval, Reason)
     ;
-        Instr1 = discard_ticket,
-        Instr = Instr1
-    ;
-        Instr1 = prune_ticket,
-        Instr = Instr1
-    ;
-        Instr1 = mark_ticket_stack(Lval1),
-        standardize_lval(Lval1, Lval),
+        Instr0 = mark_ticket_stack(Lval0),
+        standardize_lval(Lval0, Lval),
         Instr = mark_ticket_stack(Lval)
     ;
-        Instr1 = prune_tickets_to(Rval1),
-        standardize_rval(Rval1, Rval),
+        Instr0 = prune_tickets_to(Rval0),
+        standardize_rval(Rval0, Rval),
         Instr = prune_tickets_to(Rval)
     ;
-        Instr1 = incr_sp(_, _, _),
-        Instr = Instr1
-    ;
-        Instr1 = decr_sp(_),
-        Instr = Instr1
-    ;
-        Instr1 = decr_sp_and_return(_),
-        Instr = Instr1
-    ;
-        Instr1 = fork(_),
-        Instr = Instr1
-    ;
-        Instr1 = init_sync_term(Lval1, N),
-        standardize_lval(Lval1, Lval),
+        Instr0 = init_sync_term(Lval0, N),
+        standardize_lval(Lval0, Lval),
         Instr = init_sync_term(Lval, N)
     ;
-        Instr1 = join_and_continue(Lval1, Label),
-        standardize_lval(Lval1, Lval),
+        Instr0 = join_and_continue(Lval0, Label),
+        standardize_lval(Lval0, Lval),
         Instr = join_and_continue(Lval, Label)
     ;
-        Instr1 = foreign_proc_code(_, _, _, _, _, _, _, _, _),
-        Instr = Instr1
+        ( Instr0 = comment(_)
+        ; Instr0 = livevals(_)
+        ; Instr0 = block(_, _, _)
+        ; Instr0 = llcall(_, _, _, _, _, _)
+        ; Instr0 = mkframe(_, _)
+        ; Instr0 = label(_)
+        ; Instr0 = goto(_)
+        ; Instr0 = computed_goto(_, _)
+        ; Instr0 = arbitrary_c_code(_, _, _)
+        ; Instr0 = push_region_frame(_, _)
+        ; Instr0 = use_and_maybe_pop_region_frame(_, _)
+        ; Instr0 = discard_ticket
+        ; Instr0 = prune_ticket
+        ; Instr0 = incr_sp(_, _, _)
+        ; Instr0 = decr_sp(_)
+        ; Instr0 = decr_sp_and_return(_)
+        ; Instr0 = fork(_)
+        ; Instr0 = foreign_proc_code(_, _, _, _, _, _, _, _, _)
+        ),
+        Instr = Instr0
     ).
 
     % Compute the standard form of an lval.
@@ -476,32 +466,29 @@
     %
 :- pred standardize_rval(rval::in, rval::out) is det.
 
-standardize_rval(Rval1, Rval) :-
+standardize_rval(Rval0, Rval) :-
     (
-        Rval1 = lval(Lval1),
-        standardize_lval(Lval1, Lval),
+        Rval0 = lval(Lval0),
+        standardize_lval(Lval0, Lval),
         Rval = lval(Lval)
     ;
-        Rval1 = var(_),
-        unexpected(this_file, "var in standardize_rval")
-    ;
-        Rval1 = mkword(_, _),
-        Rval = Rval1
-    ;
-        Rval1 = const(_),
-        Rval = Rval1
+        ( Rval0 = mkword(_, _)
+        ; Rval0 = const(_)
+        ; Rval0 = mem_addr(_)
+        ),
+        Rval = Rval0
     ;
-        Rval1 = unop(Unop, Rval1L),
-        standardize_rval(Rval1L, RvalL),
+        Rval0 = unop(Unop, Rval0L),
+        standardize_rval(Rval0L, RvalL),
         Rval = unop(Unop, RvalL)
     ;
-        Rval1 = binop(Binop, Rval1L, Rval1R),
-        standardize_rval(Rval1L, RvalL),
-        standardize_rval(Rval1R, RvalR),
+        Rval0 = binop(Binop, Rval0L, Rval0R),
+        standardize_rval(Rval0L, RvalL),
+        standardize_rval(Rval0R, RvalR),
         Rval = binop(Binop, RvalL, RvalR)
     ;
-        Rval1 = mem_addr(_),
-        Rval = Rval1
+        Rval0 = var(_),
+        unexpected(this_file, "var in standardize_rval")
     ).
 
 %-----------------------------------------------------------------------------%
@@ -559,50 +546,50 @@
 :- pred most_specific_instrs(list(instruction)::in, list(instruction)::in,
     list(instruction)::out) is semidet.
 
-most_specific_instrs(Instrs1, Instrs2, Instrs) :-
+most_specific_instrs(InstrsA, InstrsB, Instrs) :-
     (
-        Instrs1 = [Instr1 | Tail1],
-        Instrs2 = [Instr2 | Tail2]
+        InstrsA = [InstrA | TailA],
+        InstrsB = [InstrB | TailB]
     ->
-        Instr1 = llds_instr(Uinstr1, Comment1),
-        Instr2 = llds_instr(Uinstr2, Comment2),
+        InstrA = llds_instr(UinstrA, CommentA),
+        InstrB = llds_instr(UinstrB, CommentB),
         (
-            most_specific_instr(Uinstr1, Uinstr2, yes(Uinstr))
+            most_specific_instr(UinstrA, UinstrB, yes(Uinstr))
         ->
-            ( Comment1 = Comment2 ->
-                Comment = Comment1
+            ( CommentA = CommentB ->
+                Comment = CommentA
             ;
                 Comment = "unified intruction"
             ),
             Instr = llds_instr(Uinstr, Comment),
-            most_specific_instrs(Tail1, Tail2, Tail),
+            most_specific_instrs(TailA, TailB, Tail),
             Instrs = [Instr | Tail]
         ;
-            Uinstr1 = comment(_)
+            UinstrA = comment(_)
         ->
-            most_specific_instrs(Tail1, Instrs2, Instrs)
+            most_specific_instrs(TailA, InstrsB, Instrs)
         ;
-            Uinstr2 = comment(_)
+            UinstrB = comment(_)
         ->
-            most_specific_instrs(Instrs1, Tail2, Instrs)
+            most_specific_instrs(InstrsA, TailB, Instrs)
         ;
             fail
         )
     ;
-        Instrs1 = [],
-        Instrs2 = []
+        InstrsA = [],
+        InstrsB = []
     ->
         Instrs = []
     ;
-        Instrs1 = [Instr1 | Tail1],
-        Instr1 = llds_instr(comment(_), _)
+        InstrsA = [InstrA | TailA],
+        InstrA = llds_instr(comment(_), _)
     ->
-        most_specific_instrs(Tail1, Instrs2, Instrs)
+        most_specific_instrs(TailA, InstrsB, Instrs)
     ;
-        Instrs2 = [Instr2 | Tail2],
-        Instr2 = llds_instr(comment(_), _)
+        InstrsB = [InstrB | TailB],
+        InstrB = llds_instr(comment(_), _)
     ->
-        most_specific_instrs(Instrs1, Tail2, Instrs)
+        most_specific_instrs(InstrsA, TailB, Instrs)
     ;
         fail
     ).
@@ -612,155 +599,209 @@
     %
 :- pred most_specific_instr(instr::in, instr::in, maybe(instr)::out) is det.
 
-most_specific_instr(Instr1, Instr2, MaybeInstr) :-
+most_specific_instr(InstrA, InstrB, MaybeInstr) :-
     (
-        Instr1 = assign(Lval1, Rval1),
+        InstrA = assign(LvalA, RvalA),
         (
-            Instr2 = assign(Lval2, Rval2),
-            most_specific_lval(Lval1, Lval2, Lval),
-            most_specific_rval(Rval1, Rval2, Rval)
+            InstrB = assign(LvalB, RvalB),
+            most_specific_lval(LvalA, LvalB, Lval),
+            most_specific_rval(RvalA, RvalB, Rval)
         ->
             MaybeInstr = yes(assign(Lval, Rval))
         ;
             MaybeInstr = no
         )
     ;
-        Instr1 = if_val(Rval1, CodeAddr1),
+        InstrA = keep_assign(LvalA, RvalA),
+        (
+            InstrB = keep_assign(LvalB, RvalB),
+            most_specific_lval(LvalA, LvalB, Lval),
+            most_specific_rval(RvalA, RvalB, Rval)
+        ->
+            MaybeInstr = yes(keep_assign(Lval, Rval))
+        ;
+            MaybeInstr = no
+        )
+    ;
+        InstrA = if_val(RvalA, CodeAddrA),
         (
-            Instr2 = if_val(Rval2, CodeAddr2),
-            most_specific_rval(Rval1, Rval2, Rval),
-            CodeAddr1 = CodeAddr2
+            InstrB = if_val(RvalB, CodeAddrB),
+            most_specific_rval(RvalA, RvalB, Rval),
+            CodeAddrA = CodeAddrB
         ->
-            MaybeInstr = yes(if_val(Rval, CodeAddr1))
+            MaybeInstr = yes(if_val(Rval, CodeAddrA))
         ;
             MaybeInstr = no
         )
     ;
-        Instr1 = incr_hp(Lval1, MaybeTag1, MaybeOffset1, Rval1, Msg1,
-            MayUseAtomic1, MaybeRegionRval1),
+        InstrA = incr_hp(LvalA, MaybeTag, MaybeOffset, RvalA, Msg,
+            MayUseAtomic, MaybeRegionRvalA),
         (
-            Instr2 = incr_hp(Lval2, MaybeTag2, MaybeOffset2, Rval2, Msg2,
-                MayUseAtomic2, MaybeRegionRval2),
-            most_specific_lval(Lval1, Lval2, Lval),
-            most_specific_rval(Rval1, Rval2, Rval),
-            MaybeTag1 = MaybeTag2,
-            MaybeOffset1 = MaybeOffset2,
-            Msg1 = Msg2,
-            MayUseAtomic1 = MayUseAtomic2,
+            InstrB = incr_hp(LvalB, MaybeTag, MaybeOffset, RvalB, Msg,
+                MayUseAtomic, MaybeRegionRvalB),
+            most_specific_lval(LvalA, LvalB, Lval),
+            most_specific_rval(RvalA, RvalB, Rval),
             (
-                MaybeRegionRval1 = yes(RegionRval1),
-                MaybeRegionRval2 = yes(RegionRval2),
-                most_specific_rval(RegionRval1, RegionRval2, RegionRval),
+                MaybeRegionRvalA = yes(RegionRvalA),
+                MaybeRegionRvalB = yes(RegionRvalB),
+                most_specific_rval(RegionRvalA, RegionRvalB, RegionRval),
                 MaybeRegionRval = yes(RegionRval)
             ;
-                MaybeRegionRval1 = no,
-                MaybeRegionRval2 = no,
+                MaybeRegionRvalA = no,
+                MaybeRegionRvalB = no,
                 MaybeRegionRval = no
             )
         ->
-            MaybeInstr = yes(incr_hp(Lval, MaybeTag1, MaybeOffset1, Rval,
-                Msg1, MayUseAtomic1, MaybeRegionRval))
+            MaybeInstr = yes(incr_hp(Lval, MaybeTag, MaybeOffset, Rval,
+                Msg, MayUseAtomic, MaybeRegionRval))
         ;
             MaybeInstr = no
         )
     ;
-        Instr1 = mark_hp(Lval1),
+        InstrA = mark_hp(LvalA),
         (
-            Instr2 = mark_hp(Lval2),
-            most_specific_lval(Lval1, Lval2, Lval)
+            InstrB = mark_hp(LvalB),
+            most_specific_lval(LvalA, LvalB, Lval)
         ->
             MaybeInstr = yes(mark_hp(Lval))
         ;
             MaybeInstr = no
         )
     ;
-        Instr1 = restore_hp(Rval1),
+        InstrA = restore_hp(RvalA),
         (
-            Instr2 = restore_hp(Rval2),
-            most_specific_rval(Rval1, Rval2, Rval)
+            InstrB = restore_hp(RvalB),
+            most_specific_rval(RvalA, RvalB, Rval)
         ->
             MaybeInstr = yes(restore_hp(Rval))
         ;
             MaybeInstr = no
         )
     ;
-        Instr1 = free_heap(Rval1),
+        InstrA = free_heap(RvalA),
         (
-            Instr2 = free_heap(Rval2),
-            most_specific_rval(Rval1, Rval2, Rval)
+            InstrB = free_heap(RvalB),
+            most_specific_rval(RvalA, RvalB, Rval)
         ->
             MaybeInstr = yes(free_heap(Rval))
         ;
             MaybeInstr = no
         )
     ;
-        Instr1 = store_ticket(Lval1),
+        InstrA = push_region_frame(StackId, EmbeddedStackFrame),
         (
-            Instr2 = store_ticket(Lval2),
-            most_specific_lval(Lval1, Lval2, Lval)
+            InstrB = push_region_frame(StackId, EmbeddedStackFrame)
+        ->
+            MaybeInstr = yes(push_region_frame(StackId, EmbeddedStackFrame))
+        ;
+            MaybeInstr = no
+        )
+    ;
+        InstrA = region_fill_frame(FillOp, EmbeddedStackFrame,
+            IdRvalA, NumLvalA, AddrLvalA),
+        (
+            InstrB = region_fill_frame(FillOp, EmbeddedStackFrame,
+                IdRvalB, NumLvalB, AddrLvalB),
+            most_specific_rval(IdRvalA, IdRvalB, IdRval),
+            most_specific_lval(NumLvalA, NumLvalB, NumLval),
+            most_specific_lval(AddrLvalA, AddrLvalB, AddrLval)
+        ->
+            MaybeInstr = yes(region_fill_frame(FillOp, EmbeddedStackFrame,
+                IdRval, NumLval, AddrLval))
+        ;
+            MaybeInstr = no
+        )
+    ;
+        InstrA = region_set_fixed_slot(SetOp, EmbeddedStackFrame,
+            ValueRvalA),
+        (
+            InstrB = region_set_fixed_slot(SetOp, EmbeddedStackFrame,
+                ValueRvalB),
+            most_specific_rval(ValueRvalA, ValueRvalB, ValueRval)
+        ->
+            MaybeInstr = yes(region_set_fixed_slot(SetOp, EmbeddedStackFrame,
+                ValueRval))
+        ;
+            MaybeInstr = no
+        )
+    ;
+        InstrA = use_and_maybe_pop_region_frame(UseOp, EmbeddedStackFrame),
+        (
+            InstrB = use_and_maybe_pop_region_frame(UseOp, EmbeddedStackFrame)
+        ->
+            MaybeInstr = yes(use_and_maybe_pop_region_frame(UseOp,
+                EmbeddedStackFrame))
+        ;
+            MaybeInstr = no
+        )
+    ;
+        InstrA = store_ticket(LvalA),
+        (
+            InstrB = store_ticket(LvalB),
+            most_specific_lval(LvalA, LvalB, Lval)
         ->
             MaybeInstr = yes(store_ticket(Lval))
         ;
             MaybeInstr = no
         )
     ;
-        Instr1 = reset_ticket(Rval1, Reason),
+        InstrA = reset_ticket(RvalA, Reason),
         (
-            Instr2 = reset_ticket(Rval2, Reason),
-            most_specific_rval(Rval1, Rval2, Rval)
+            InstrB = reset_ticket(RvalB, Reason),
+            most_specific_rval(RvalA, RvalB, Rval)
         ->
             MaybeInstr = yes(reset_ticket(Rval, Reason))
         ;
             MaybeInstr = no
         )
     ;
-        Instr1 = mark_ticket_stack(Lval1),
+        InstrA = mark_ticket_stack(LvalA),
         (
-            Instr2 = mark_ticket_stack(Lval2),
-            most_specific_lval(Lval1, Lval2, Lval)
+            InstrB = mark_ticket_stack(LvalB),
+            most_specific_lval(LvalA, LvalB, Lval)
         ->
             MaybeInstr = yes(mark_ticket_stack(Lval))
         ;
             MaybeInstr = no
         )
     ;
-        Instr1 = prune_tickets_to(Rval1),
+        InstrA = prune_tickets_to(RvalA),
         (
-            Instr2 = prune_tickets_to(Rval2),
-            most_specific_rval(Rval1, Rval2, Rval)
+            InstrB = prune_tickets_to(RvalB),
+            most_specific_rval(RvalA, RvalB, Rval)
         ->
             MaybeInstr = yes(prune_tickets_to(Rval))
         ;
             MaybeInstr = no
         )
     ;
-        ( Instr1 = livevals(_)
-        ; Instr1 = block(_, _, _)
-        ; Instr1 = llcall(_, _, _, _, _, _)
-        ; Instr1 = mkframe(_, _)
-        ; Instr1 = label(_)
-        ; Instr1 = goto(_)
-        ; Instr1 = computed_goto(_, _)
-        ; Instr1 = arbitrary_c_code(_, _, _)
-        ; Instr1 = save_maxfr(_)
-        ; Instr1 = restore_maxfr(_)
-        ; Instr1 = discard_ticket
-        ; Instr1 = prune_ticket
-        ; Instr1 = incr_sp(_, _, _)
-        ; Instr1 = decr_sp(_)
-        ; Instr1 = decr_sp_and_return(_)
-        ; Instr1 = foreign_proc_code(_, _, _, _, _, _, _, _, _)
-        ; Instr1 = fork(_)
-        ; Instr1 = init_sync_term(_, _)
-        ; Instr1 = join_and_continue(_, _)
+        ( InstrA = livevals(_)
+        ; InstrA = block(_, _, _)
+        ; InstrA = llcall(_, _, _, _, _, _)
+        ; InstrA = mkframe(_, _)
+        ; InstrA = label(_)
+        ; InstrA = goto(_)
+        ; InstrA = computed_goto(_, _)
+        ; InstrA = arbitrary_c_code(_, _, _)
+        ; InstrA = save_maxfr(_)
+        ; InstrA = restore_maxfr(_)
+        ; InstrA = discard_ticket
+        ; InstrA = prune_ticket
+        ; InstrA = incr_sp(_, _, _)
+        ; InstrA = decr_sp(_)
+        ; InstrA = decr_sp_and_return(_)
+        ; InstrA = foreign_proc_code(_, _, _, _, _, _, _, _, _)
+        ; InstrA = fork(_)
+        ; InstrA = init_sync_term(_, _)
+        ; InstrA = join_and_continue(_, _)
         ),
-        ( Instr1 = Instr2 ->
-            MaybeInstr = yes(Instr1)
+        ( InstrA = InstrB ->
+            MaybeInstr = yes(InstrA)
         ;
             MaybeInstr = no
         )
     ;
-        Instr1 = comment(_),
+        InstrA = comment(_),
         MaybeInstr = no
     ).
 
@@ -769,86 +810,39 @@
     %
 :- pred most_specific_lval(lval::in, lval::in, lval::out) is semidet.
 
-most_specific_lval(Lval1, Lval2, Lval) :-
+most_specific_lval(LvalA, LvalB, Lval) :-
     (
-        Lval1 = reg(_, _),
-        Lval2 = Lval1,
-        Lval = Lval1
-    ;
-        Lval1 = succip,
-        Lval2 = Lval1,
-        Lval = Lval1
-    ;
-        Lval1 = maxfr,
-        Lval2 = Lval1,
-        Lval = Lval1
-    ;
-        Lval1 = curfr,
-        Lval2 = Lval1,
-        Lval = Lval1
-    ;
-        Lval1 = hp,
-        Lval2 = Lval1,
-        Lval = Lval1
-    ;
-        Lval1 = sp,
-        Lval2 = Lval1,
-        Lval = Lval1
-    ;
-        Lval1 = parent_sp,
-        Lval2 = Lval1,
-        Lval = Lval1
-    ;
-        Lval1 = temp(_, _),
-        Lval2 = Lval1,
-        Lval = Lval1
-    ;
-        Lval1 = stackvar(_),
-        Lval2 = Lval1,
-        Lval = Lval1
-    ;
-        Lval1 = parent_stackvar(_),
-        Lval2 = Lval1,
-        Lval = Lval1
-    ;
-        Lval1 = framevar(_),
-        Lval2 = Lval1,
-        Lval = Lval1
-    ;
-        Lval1 = succip_slot(_),
-        Lval2 = Lval1,
-        Lval = Lval1
-    ;
-        Lval1 = redoip_slot(_),
-        Lval2 = Lval1,
-        Lval = Lval1
-    ;
-        Lval1 = redofr_slot(_),
-        Lval2 = Lval1,
-        Lval = Lval1
-    ;
-        Lval1 = succfr_slot(_),
-        Lval2 = Lval1,
-        Lval = Lval1
-    ;
-        Lval1 = prevfr_slot(_),
-        Lval2 = Lval1,
-        Lval = Lval1
-    ;
-        Lval1 = field(MaybeTag1, Addr, FieldNum),
-        Lval2 = field(MaybeTag2, Addr, FieldNum),
-        ( MaybeTag1 = MaybeTag2 ->
-            MaybeTag = MaybeTag1
+        ( LvalA = reg(_, _)
+        ; LvalA = succip
+        ; LvalA = maxfr
+        ; LvalA = curfr
+        ; LvalA = hp
+        ; LvalA = sp
+        ; LvalA = parent_sp
+        ; LvalA = temp(_, _)
+        ; LvalA = stackvar(_)
+        ; LvalA = parent_stackvar(_)
+        ; LvalA = framevar(_)
+        ; LvalA = succip_slot(_)
+        ; LvalA = redoip_slot(_)
+        ; LvalA = redofr_slot(_)
+        ; LvalA = succfr_slot(_)
+        ; LvalA = prevfr_slot(_)
+        ; LvalA = mem_ref(_)
+        ),
+        LvalA = LvalB,
+        Lval = LvalA
+    ;
+        LvalA = field(MaybeTagA, Addr, FieldNum),
+        LvalB = field(MaybeTagB, Addr, FieldNum),
+        ( MaybeTagA = MaybeTagB ->
+            MaybeTag = MaybeTagA
         ;
             MaybeTag = no
         ),
         Lval = field(MaybeTag, Addr, FieldNum)
     ;
-        Lval1 = mem_ref(_),
-        Lval2 = Lval1,
-        Lval = Lval1
-    ;
-        Lval1 = lvar(_),
+        LvalA = lvar(_),
         unexpected(this_file, "lvar in most_specific_lval")
     ).
 
@@ -857,38 +851,33 @@
     %
 :- pred most_specific_rval(rval::in, rval::in, rval::out) is semidet.
 
-most_specific_rval(Rval1, Rval2, Rval) :-
+most_specific_rval(RvalA, RvalB, Rval) :-
     (
-        Rval1 = lval(Lval1),
-        Rval2 = lval(Lval2),
-        most_specific_lval(Lval1, Lval2, Lval),
+        RvalA = lval(LvalA),
+        RvalB = lval(LvalB),
+        most_specific_lval(LvalA, LvalB, Lval),
         Rval = lval(Lval)
     ;
-        Rval1 = var(_),
+        RvalA = var(_),
         unexpected(this_file, "var in most_specific_rval")
     ;
-        Rval1 = mkword(_, _),
-        Rval2 = Rval1,
-        Rval = Rval1
-    ;
-        Rval1 = const(_),
-        Rval2 = Rval1,
-        Rval = Rval1
-    ;
-        Rval1 = unop(Unop, Rval1L),
-        Rval2 = unop(Unop, Rval2L),
-        most_specific_rval(Rval1L, Rval2L, RvalL),
+        ( RvalA = mkword(_, _)
+        ; RvalA = const(_)
+        ; RvalA = mem_addr(_)
+        ),
+        RvalB = RvalA,
+        Rval = RvalA
+    ;
+        RvalA = unop(Unop, RvalAL),
+        RvalB = unop(Unop, RvalBL),
+        most_specific_rval(RvalAL, RvalBL, RvalL),
         Rval = unop(Unop, RvalL)
     ;
-        Rval1 = binop(Binnop, Rval1L, Rval1R),
-        Rval2 = binop(Binnop, Rval2L, Rval2R),
-        most_specific_rval(Rval1L, Rval2L, RvalL),
-        most_specific_rval(Rval1R, Rval2R, RvalR),
+        RvalA = binop(Binnop, RvalAL, RvalAR),
+        RvalB = binop(Binnop, RvalBL, RvalBR),
+        most_specific_rval(RvalAL, RvalBL, RvalL),
+        most_specific_rval(RvalAR, RvalBR, RvalR),
         Rval = binop(Binnop, RvalL, RvalR)
-    ;
-        Rval1 = mem_addr(_),
-        Rval2 = Rval1,
-        Rval = Rval1
     ).
 
 %-----------------------------------------------------------------------------%
Index: compiler/dupproc.m
===================================================================
RCS file: /home/mercury/mercury1/repository/mercury/compiler/dupproc.m,v
retrieving revision 1.19
diff -u -b -r1.19 dupproc.m
--- compiler/dupproc.m	9 Jul 2007 13:28:31 -0000	1.19
+++ compiler/dupproc.m	30 Jul 2007 04:50:29 -0000
@@ -172,12 +172,6 @@
 
 standardize_instr(Instr, StdInstr, DupProcMap) :-
     (
-        Instr = comment(_),
-        StdInstr = comment("")
-    ;
-        Instr = livevals(_),
-        StdInstr = Instr
-    ;
         Instr = block(NumIntTemps, NumFloatTemps, Instrs),
         standardize_instrs(Instrs, StdInstrs, DupProcMap),
         StdInstr = block(NumIntTemps, NumFloatTemps, StdInstrs)
@@ -186,6 +180,10 @@
         standardize_rval(Rval, StdRval, DupProcMap),
         StdInstr = assign(Lval, StdRval)
     ;
+        Instr = keep_assign(Lval, Rval),
+        standardize_rval(Rval, StdRval, DupProcMap),
+        StdInstr = keep_assign(Lval, StdRval)
+    ;
         Instr = llcall(Target, Cont, LiveInfo, Context, GoalPath, Model),
         standardize_code_addr(Target, StdTarget, DupProcMap),
         standardize_code_addr(Cont, StdCont, DupProcMap),
@@ -216,73 +214,52 @@
         standardize_labels(Targets, StdTargets, DupProcMap),
         StdInstr = computed_goto(Rval, StdTargets)
     ;
-        Instr = arbitrary_c_code(_, _, _),
-        StdInstr = Instr
-    ;
-        Instr = save_maxfr(_),
-        StdInstr = Instr
-    ;
-        Instr = restore_maxfr(_),
-        StdInstr = Instr
-    ;
         Instr = if_val(Rval, Target),
         standardize_rval(Rval, StdRval, DupProcMap),
         standardize_code_addr(Target, StdTarget, DupProcMap),
         StdInstr = if_val(StdRval, StdTarget)
     ;
-        Instr = incr_hp(_, _, _, _, _, _, _),
-        StdInstr = Instr
-    ;
-        Instr = mark_hp(_),
-        StdInstr = Instr
-    ;
-        Instr = restore_hp(_),
-        StdInstr = Instr
-    ;
-        Instr = free_heap(_),
-        StdInstr = Instr
-    ;
-        Instr = store_ticket(_),
-        StdInstr = Instr
-    ;
-        Instr = reset_ticket(_, _),
-        StdInstr = Instr
-    ;
-        Instr = discard_ticket,
-        StdInstr = Instr
-    ;
-        Instr = prune_ticket,
-        StdInstr = Instr
-    ;
-        Instr = mark_ticket_stack(_),
-        StdInstr = Instr
-    ;
-        Instr = prune_tickets_to(_),
-        StdInstr = Instr
-    ;
         Instr = incr_sp(NumSlots, _, Kind),
         StdInstr = incr_sp(NumSlots, "", Kind)
     ;
-        Instr = decr_sp(_),
-        StdInstr = Instr
-    ;
-        Instr = decr_sp_and_return(_),
-        StdInstr = Instr
-    ;
         Instr = fork(Child),
         standardize_label(Child, StdChild, DupProcMap),
         StdInstr = fork(StdChild)
     ;
-        Instr = init_sync_term(_, _),
-        StdInstr = Instr
-    ;
         Instr = join_and_continue(Lval, Label),
         standardize_label(Label, StdLabel, DupProcMap),
         StdInstr = join_and_continue(Lval, StdLabel)
     ;
-        Instr = foreign_proc_code(_, _, _, _, _, _, _, _, _),
         % The labels occurring in foreign_proc_code instructions
         % cannot be substituted.
+        Instr = foreign_proc_code(_, _, _, _, _, _, _, _, _),
+        StdInstr = Instr
+    ;
+        % These instructions have no labels inside them, or anything else
+        % that can be standardized.
+        ( Instr = comment(_)
+        ; Instr = livevals(_)
+        ; Instr = arbitrary_c_code(_, _, _)
+        ; Instr = save_maxfr(_)
+        ; Instr = restore_maxfr(_)
+        ; Instr = incr_hp(_, _, _, _, _, _, _)
+        ; Instr = mark_hp(_)
+        ; Instr = restore_hp(_)
+        ; Instr = free_heap(_)
+        ; Instr = push_region_frame(_, _)
+        ; Instr = region_fill_frame(_, _, _, _, _)
+        ; Instr = region_set_fixed_slot(_, _, _)
+        ; Instr = use_and_maybe_pop_region_frame(_, _)
+        ; Instr = store_ticket(_)
+        ; Instr = reset_ticket(_, _)
+        ; Instr = discard_ticket
+        ; Instr = prune_ticket
+        ; Instr = mark_ticket_stack(_)
+        ; Instr = prune_tickets_to(_)
+        ; Instr = decr_sp(_)
+        ; Instr = decr_sp_and_return(_)
+        ; Instr = init_sync_term(_, _)
+        ),
         StdInstr = Instr
     ).
 
Index: compiler/exprn_aux.m
===================================================================
RCS file: /home/mercury/mercury1/repository/mercury/compiler/exprn_aux.m,v
retrieving revision 1.82
diff -u -b -r1.82 exprn_aux.m
--- compiler/exprn_aux.m	9 Jul 2007 13:28:31 -0000	1.82
+++ compiler/exprn_aux.m	30 Jul 2007 04:51:41 -0000
@@ -372,6 +372,11 @@
         transform_lval_in_rval(Transform, Rval0, Rval, !Acc),
         Uinstr = assign(Lval, Rval)
     ;
+        Uinstr0 = keep_assign(Lval0, Rval0),
+        Transform(Lval0, Lval, !Acc),
+        transform_lval_in_rval(Transform, Rval0, Rval, !Acc),
+        Uinstr = keep_assign(Lval, Rval)
+    ;
         Uinstr0 = computed_goto(Rval0, Labels),
         transform_lval_in_rval(Transform, Rval0, Rval, !Acc),
         Uinstr = computed_goto(Rval, Labels)
@@ -420,6 +425,26 @@
         transform_lval_in_rval(Transform, Rval0, Rval, !Acc),
         Uinstr = free_heap(Rval)
     ;
+        Uinstr0 = push_region_frame(StackId, EmbeddedStackFrame),
+        Uinstr = push_region_frame(StackId, EmbeddedStackFrame)
+    ;
+        Uinstr0 = region_fill_frame(FillOp, EmbeddedStackFrame, IdRval0,
+            NumLval0, AddrLval0),
+        transform_lval_in_rval(Transform, IdRval0, IdRval, !Acc),
+        Transform(NumLval0, NumLval, !Acc),
+        Transform(AddrLval0, AddrLval, !Acc),
+        Uinstr = region_fill_frame(FillOp, EmbeddedStackFrame, IdRval,
+            NumLval, AddrLval)
+    ;
+        Uinstr0 = region_set_fixed_slot(SetOp, EmbeddedStackFrame,
+            ValueRval0),
+        transform_lval_in_rval(Transform, ValueRval0, ValueRval, !Acc),
+        Uinstr = region_set_fixed_slot(SetOp, EmbeddedStackFrame,
+            ValueRval)
+    ;
+        Uinstr0 = use_and_maybe_pop_region_frame(UseOp, EmbeddedStackFrame),
+        Uinstr = use_and_maybe_pop_region_frame(UseOp, EmbeddedStackFrame)
+    ;
         Uinstr0 = store_ticket(Lval0),
         Transform(Lval0, Lval, !Acc),
         Uinstr = store_ticket(Lval)
Index: compiler/frameopt.m
===================================================================
RCS file: /home/mercury/mercury1/repository/mercury/compiler/frameopt.m,v
retrieving revision 1.111
diff -u -b -r1.111 frameopt.m
--- compiler/frameopt.m	6 Jan 2007 09:23:32 -0000	1.111
+++ compiler/frameopt.m	23 Jul 2007 15:15:47 -0000
@@ -1199,24 +1199,24 @@
         (
             LastUinstr0 = goto(GotoTarget0)
         ->
-            replace_labels_code_addr(GotoTarget0, PreExitDummyLabelMap,
-                GotoTarget),
+            replace_labels_code_addr(GotoTarget0, GotoTarget,
+                PreExitDummyLabelMap),
             LastUinstr = goto(GotoTarget),
             LastInstr = llds_instr(LastUinstr, Comment),
             BlockInstrs = AllButLastInstrs ++ [LastInstr]
         ;
             LastUinstr0 = if_val(Rval, GotoTarget0)
         ->
-            replace_labels_code_addr(GotoTarget0, PreExitDummyLabelMap,
-                GotoTarget),
+            replace_labels_code_addr(GotoTarget0, GotoTarget,
+                PreExitDummyLabelMap),
             LastUinstr = if_val(Rval, GotoTarget),
             LastInstr = llds_instr(LastUinstr, Comment),
             BlockInstrs = AllButLastInstrs ++ [LastInstr]
         ;
             LastUinstr0 = computed_goto(Rval, GotoTargets0)
         ->
-            replace_labels_label_list(GotoTargets0, PreExitDummyLabelMap,
-                GotoTargets),
+            replace_labels_label_list(GotoTargets0, GotoTargets, 
+                PreExitDummyLabelMap),
             LastUinstr = computed_goto(Rval, GotoTargets),
             LastInstr = llds_instr(LastUinstr, Comment),
             BlockInstrs = AllButLastInstrs ++ [LastInstr]
@@ -1230,9 +1230,9 @@
                 Comps0 = Comps
             ;
                 NF0 = yes(NFLabel0),
-                replace_labels_label(NFLabel0, PreExitDummyLabelMap, NFLabel),
+                replace_labels_label(NFLabel0, NFLabel, PreExitDummyLabelMap),
                 NF = yes(NFLabel),
-                replace_labels_comps(Comps0, PreExitDummyLabelMap, Comps)
+                replace_labels_comps(Comps0, Comps, PreExitDummyLabelMap)
             ),
             LastUinstr = foreign_proc_code(D, Comps, MC, FNL, FL, FOL, NF,
                 S, MD),
@@ -1952,7 +1952,7 @@
     ),
     list.split_last_det(Instrs0, PrevInstrs, LastInstr0),
     map.from_assoc_list(AssocLabelMap, LabelMap),
-    opt_util.replace_labels_instruction(LastInstr0, LabelMap, no, LastInstr),
+    opt_util.replace_labels_instruction(LastInstr0, LastInstr, LabelMap, no),
     Instrs = PrevInstrs ++ [LastInstr | RedirectFallThrough],
     BlockInfo = frame_block_info(Label0, Instrs, FallInto,
         SideLabels, MaybeFallThrough, Type),
Index: compiler/global_data.m
===================================================================
RCS file: /home/mercury/mercury1/repository/mercury/compiler/global_data.m,v
retrieving revision 1.32
diff -u -b -r1.32 global_data.m
--- compiler/global_data.m	9 Jul 2007 13:28:31 -0000	1.32
+++ compiler/global_data.m	30 Jul 2007 04:52:39 -0000
@@ -138,6 +138,7 @@
 :- import_module counter.
 :- import_module int.
 :- import_module map.
+:- import_module maybe.
 :- import_module pair.
 :- import_module require.
 :- import_module svbimap.
@@ -691,10 +692,10 @@
 
 merge_cell_type_num_maps(CellType, BTypeNum,
         !TypeCounter, !CellTypeNumMap, !TypeNumRemap) :-
-    (if bimap.search(!.CellTypeNumMap, CellType, ATypeNum) then
+    ( bimap.search(!.CellTypeNumMap, CellType, ATypeNum) ->
         % A type also in GlobalDataA.
         svmap.det_insert(BTypeNum, ATypeNum, !TypeNumRemap)
-    else
+    ;
         % A type not in GlobalDataA.
         counter.allocate(N, !TypeCounter),
         NewTypeNum = type_num(N),
@@ -725,11 +726,9 @@
 merge_scalar_cell_group_maps_2(TypeNumRemap, BTypeNum, BScalarCellGroup,
         !ScalarCellGroupMap, !Remap) :-
     map.lookup(TypeNumRemap, BTypeNum, TypeNum),
-    (if
-        ScalarCellGroupPrime = !.ScalarCellGroupMap ^ elem(TypeNum)
-    then
+    ( map.search(!.ScalarCellGroupMap, TypeNum, ScalarCellGroupPrime) ->
         ScalarCellGroup0 = ScalarCellGroupPrime
-    else
+    ;
         % Could do this more efficiently.
         ScalarCellGroup0 = scalar_cell_group(counter.init(0), bimap.init, [])
     ),
@@ -766,12 +765,10 @@
 
 merge_scalar_cell_groups_2(TypeNum, ArrayB, ArrayAB,
         Rvals, BDataName, !GroupMembers, !GroupRemap) :-
-    (if
-        bimap.search(!.GroupMembers, Rvals, DataName)
-    then
+    ( bimap.search(!.GroupMembers, Rvals, DataName) ->
         % Seen this list of rvals before in the group.
         svmap.det_insert(BDataName, DataName, !GroupRemap)
-    else
+    ;
         % Not seen this list of rvals before in the group.
         (
             BDataName = scalar_common_ref(_, BCellNum),
@@ -820,15 +817,16 @@
     % The scalar cell group and vector cell group contents themselves
     % need to be updated to use the merged cell information.
     %
-:- func remap_static_cell_info(static_cell_remap_info,
-    static_cell_info) = static_cell_info.
+:- func remap_static_cell_info(static_cell_remap_info, static_cell_info)
+    = static_cell_info.
 
 remap_static_cell_info(Remap, SCI0) = SCI :-
     ScalarMap = map.map_values(remap_scalar_cell_group(Remap),
         SCI0 ^ scalar_cell_group_map),
     VectorMap = map.map_values(remap_vector_cell_group(Remap),
         SCI0 ^ vector_cell_group_map),
-    SCI = (SCI0 ^ scalar_cell_group_map := ScalarMap)
+    SCI = (SCI0
+        ^ scalar_cell_group_map := ScalarMap)
                 ^ vector_cell_group_map := VectorMap.
 
 :- func remap_scalar_cell_group(static_cell_remap_info,
@@ -836,24 +834,24 @@
 
 remap_scalar_cell_group(Remap, _, ScalarCellGroup0) = ScalarCellGroup :-
     Array0 = ScalarCellGroup0 ^ scalar_cell_rev_array,
-    Array  = list.map(remap_common_cell_value(Remap), Array0),
-    ScalarCellGroup = ScalarCellGroup0 ^ scalar_cell_rev_array := Array.
+    ScalarCellGroup = ScalarCellGroup0 ^ scalar_cell_rev_array := Array,
+    Array = list.map(remap_common_cell_value(Remap), Array0).
 
 :- func remap_vector_cell_group(static_cell_remap_info,
     type_num, vector_cell_group) = vector_cell_group.
 
 remap_vector_cell_group(Remap, _, VectorCellGroup0) = VectorCellGroup :-
     VectorCellGroup0 = vector_cell_group(Counter, Map0),
-    VectorCellGroup  = vector_cell_group(Counter, Map),
-    Map = map.map_values(remap_vector_contents(Remap), Map0).
+    Map = map.map_values(remap_vector_contents(Remap), Map0),
+    VectorCellGroup = vector_cell_group(Counter, Map).
 
 :- func remap_vector_contents(static_cell_remap_info,
     int, vector_contents) = vector_contents.
 
 remap_vector_contents(Remap, _, Contents0) = Contents :-
     Contents0 = vector_contents(Values0),
-    Contents  = vector_contents(Values),
-    Values = list.map(remap_common_cell_value(Remap), Values0).
+    Values = list.map(remap_common_cell_value(Remap), Values0),
+    Contents = vector_contents(Values).
 
 :- func remap_common_cell_value(static_cell_remap_info,
     common_cell_value) = common_cell_value.
@@ -861,12 +859,12 @@
 remap_common_cell_value(Remap, CommonCellValue0) = CommonCellValue :-
     (
         CommonCellValue0 = plain_value(RvalsTypes0),
-        CommonCellValue  = plain_value(RvalsTypes),
-        RvalsTypes = list.map(remap_plain_value(Remap), RvalsTypes0)
+        RvalsTypes = list.map(remap_plain_value(Remap), RvalsTypes0),
+        CommonCellValue = plain_value(RvalsTypes)
     ;
         CommonCellValue0 = grouped_args_value(ArgGroup0),
-        CommonCellValue  = grouped_args_value(ArgGroup),
-        ArgGroup = list.map(remap_arg_group_value(Remap), ArgGroup0)
+        ArgGroup = list.map(remap_arg_group_value(Remap), ArgGroup0),
+        CommonCellValue = grouped_args_value(ArgGroup)
     ).
 
 :- func remap_plain_value(static_cell_remap_info,
@@ -881,12 +879,12 @@
 remap_arg_group_value(Remap, GroupedArgs0) = GroupedArgs :-
     (
         GroupedArgs0 = common_cell_grouped_args(Type, Fields, Rvals0),
-        GroupedArgs  = common_cell_grouped_args(Type, Fields, Rvals),
-        Rvals = list.map(remap_rval(Remap), Rvals0)
+        Rvals = list.map(remap_rval(Remap), Rvals0),
+        GroupedArgs = common_cell_grouped_args(Type, Fields, Rvals)
     ;
         GroupedArgs0 = common_cell_ungrouped_arg(Type, Rvals0),
-        GroupedArgs  = common_cell_ungrouped_arg(Type, Rvals),
-        Rvals = remap_rval(Remap, Rvals0)
+        Rvals = remap_rval(Remap, Rvals0),
+        GroupedArgs = common_cell_ungrouped_arg(Type, Rvals)
     ).
 
 %-----------------------------------------------------------------------------%
@@ -906,20 +904,105 @@
 remap_instr(Remap, Instr0) = Instr :-
     (
         Instr0 = block(NumIntTemps, NumFloatTemps, Block0),
-        Instr  = block(NumIntTemps, NumFloatTemps, Block),
-        Block  = list.map(remap_instruction(Remap), Block0)
+        Block = list.map(remap_instruction(Remap), Block0),
+        Instr = block(NumIntTemps, NumFloatTemps, Block)
     ;
         Instr0 = assign(Lval, Rval0),
-        Instr  = assign(Lval, Rval),
-        Rval = remap_rval(Remap, Rval0)
+        Rval = remap_rval(Remap, Rval0),
+        Instr = assign(Lval, Rval)
+    ;
+        Instr0 = keep_assign(Lval, Rval0),
+        Rval = remap_rval(Remap, Rval0),
+        Instr = keep_assign(Lval, Rval)
     ;
         Instr0 = if_val(Rval0, CodeAddr),
-        Instr  = if_val(Rval, CodeAddr),
-        Rval = remap_rval(Remap, Rval0)
+        Rval = remap_rval(Remap, Rval0),
+        Instr  = if_val(Rval, CodeAddr)
     ;
         Instr0 = foreign_proc_code(A, Comps0, B, C, D, E, F, G, H),
-        Instr  = foreign_proc_code(A, Comps,  B, C, D, E, F, G, H),
-        Comps = list.map(remap_foreign_proc_component(Remap), Comps0)
+        Comps = list.map(remap_foreign_proc_component(Remap), Comps0),
+        Instr  = foreign_proc_code(A, Comps,  B, C, D, E, F, G, H)
+    ;
+        Instr0 = computed_goto(Rval0, CodeAddrs),
+        Rval = remap_rval(Remap, Rval0),
+        Instr = computed_goto(Rval, CodeAddrs)
+    ;
+        Instr0 = save_maxfr(Lval0),
+        Lval = remap_lval(Remap, Lval0),
+        Instr = save_maxfr(Lval)
+    ;
+        Instr0 = restore_maxfr(Lval0),
+        Lval = remap_lval(Remap, Lval0),
+        Instr = restore_maxfr(Lval)
+    ;
+        Instr0 = incr_hp(Lval0, MaybeTag, MaybeOffset, SizeRval0, Prof,
+            Atomic, MaybeRegion0),
+        Lval = remap_lval(Remap, Lval0),
+        SizeRval = remap_rval(Remap, SizeRval0),
+        (
+            MaybeRegion0 = yes(Region0),
+            Region = remap_rval(Remap, Region0),
+            MaybeRegion = yes(Region)
+        ;
+            MaybeRegion0 = no,
+            MaybeRegion = no
+        ),
+        Instr = incr_hp(Lval, MaybeTag, MaybeOffset, SizeRval, Prof,
+            Atomic, MaybeRegion)
+    ;
+        Instr0 = mark_hp(Lval0),
+        Lval = remap_lval(Remap, Lval0),
+        Instr = mark_hp(Lval)
+    ;
+        Instr0 = restore_hp(Rval0),
+        Rval = remap_rval(Remap, Rval0),
+        Instr = restore_hp(Rval)
+    ;
+        Instr0 = free_heap(Rval0),
+        Rval = remap_rval(Remap, Rval0),
+        Instr = free_heap(Rval)
+    ;
+        Instr0 = push_region_frame(StackId, EmbeddedStackFrame),
+        Instr = push_region_frame(StackId, EmbeddedStackFrame)
+    ;
+        Instr0 = region_fill_frame(FillOp, EmbeddedStackFrame, IdRval0,
+            NumLval0, AddrLval0),
+        IdRval = remap_rval(Remap, IdRval0),
+        NumLval = remap_lval(Remap, NumLval0),
+        AddrLval = remap_lval(Remap, AddrLval0),
+        Instr = region_fill_frame(FillOp, EmbeddedStackFrame, IdRval,
+            NumLval, AddrLval)
+    ;
+        Instr0 = region_set_fixed_slot(SetOp, EmbeddedStackFrame, ValueRval0),
+        ValueRval = remap_rval(Remap, ValueRval0),
+        Instr = region_set_fixed_slot(SetOp, EmbeddedStackFrame, ValueRval)
+    ;
+        Instr0 = use_and_maybe_pop_region_frame(UseOp, EmbeddedStackFrame),
+        Instr = use_and_maybe_pop_region_frame(UseOp, EmbeddedStackFrame)
+    ;
+        Instr0 = store_ticket(Lval0),
+        Lval = remap_lval(Remap, Lval0),
+        Instr = store_ticket(Lval)
+    ;
+        Instr0 = reset_ticket(Rval0, Reason),
+        Rval = remap_rval(Remap, Rval0),
+        Instr = reset_ticket(Rval, Reason)
+    ;
+        Instr0 = mark_ticket_stack(Lval0),
+        Lval = remap_lval(Remap, Lval0),
+        Instr = mark_ticket_stack(Lval)
+    ;
+        Instr0 = prune_tickets_to(Rval0),
+        Rval = remap_rval(Remap, Rval0),
+        Instr = prune_tickets_to(Rval)
+    ;
+        Instr0 = init_sync_term(Lval0, NumJoins),
+        Lval = remap_lval(Remap, Lval0),
+        Instr = init_sync_term(Lval, NumJoins)
+    ;
+        Instr0 = join_and_continue(Lval0, Label),
+        Lval = remap_lval(Remap, Lval0),
+        Instr = join_and_continue(Lval, Label)
     ;
         ( Instr0 = comment(_)
         ; Instr0 = livevals(_)
@@ -927,26 +1010,13 @@
         ; Instr0 = mkframe(_, _)
         ; Instr0 = label(_)
         ; Instr0 = goto(_)
-        ; Instr0 = computed_goto(_, _)
         ; Instr0 = arbitrary_c_code(_, _, _)
-        ; Instr0 = save_maxfr(_)
-        ; Instr0 = restore_maxfr(_)
-        ; Instr0 = incr_hp(_, _, _, _, _, _, _)
-        ; Instr0 = mark_hp(_)
-        ; Instr0 = restore_hp(_)
-        ; Instr0 = free_heap(_)
-        ; Instr0 = store_ticket(_)
-        ; Instr0 = reset_ticket(_, _)
         ; Instr0 = prune_ticket
         ; Instr0 = discard_ticket
-        ; Instr0 = mark_ticket_stack(_)
-        ; Instr0 = prune_tickets_to(_)
         ; Instr0 = incr_sp(_, _, _)
         ; Instr0 = decr_sp(_)
         ; Instr0 = decr_sp_and_return(_)
-        ; Instr0 = init_sync_term(_, _)
         ; Instr0 = fork(_)
-        ; Instr0 = join_and_continue(_, _)
         ),
         Instr = Instr0
     ).
@@ -957,12 +1027,12 @@
 remap_foreign_proc_component(Remap, Comp0) = Comp :-
     (
         Comp0 = foreign_proc_inputs(Inputs0),
-        Comp  = foreign_proc_inputs(Inputs),
-        Inputs = list.map(remap_foreign_proc_input(Remap), Inputs0) 
+        Inputs = list.map(remap_foreign_proc_input(Remap), Inputs0),
+        Comp = foreign_proc_inputs(Inputs)
     ;
         Comp0 = foreign_proc_outputs(Outputs0),
-        Comp  = foreign_proc_outputs(Outputs),
-        Outputs = list.map(remap_foreign_proc_output(Remap), Outputs0)
+        Outputs = list.map(remap_foreign_proc_output(Remap), Outputs0),
+        Comp = foreign_proc_outputs(Outputs)
     ;
         ( Comp0 = foreign_proc_raw_code(_, _, _, _)
         ; Comp0 = foreign_proc_user_code(_, _, _)
@@ -975,28 +1045,30 @@
 :- func remap_foreign_proc_input(static_cell_remap_info, foreign_proc_input)
     = foreign_proc_input.
 
-remap_foreign_proc_input(Remap, foreign_proc_input(A, B, C, D, Rval0, E, F))
-        = foreign_proc_input(A, B, C, D, Rval, E, F) :-
-    Rval = remap_rval(Remap, Rval0).
+remap_foreign_proc_input(Remap, Input0) = Input :-
+    Input0 = foreign_proc_input(A, B, C, D, Rval0, E, F),
+    Rval = remap_rval(Remap, Rval0),
+    Input = foreign_proc_input(A, B, C, D, Rval, E, F).
 
 :- func remap_foreign_proc_output(static_cell_remap_info, foreign_proc_output)
     = foreign_proc_output.
 
-remap_foreign_proc_output(Remap, foreign_proc_output(Lval0, A, B, C, D, E, F))
-        = foreign_proc_output(Lval, A, B, C, D, E, F) :-
-    Lval = remap_lval(Remap, Lval0).
+remap_foreign_proc_output(Remap, Output0) = Output :-
+    Output0 = foreign_proc_output(Lval0, A, B, C, D, E, F),
+    Lval = remap_lval(Remap, Lval0),
+    Output = foreign_proc_output(Lval, A, B, C, D, E, F).
 
 :- func remap_lval(static_cell_remap_info, lval) = lval.
 
 remap_lval(Remap, Lval0) = Lval :-
     (
         Lval0 = field(MaybeTag, Rval0, FieldNum),
-        Lval  = field(MaybeTag, Rval, FieldNum),
-        Rval  = remap_rval(Remap, Rval0)
+        Rval = remap_rval(Remap, Rval0),
+        Lval = field(MaybeTag, Rval, FieldNum)
     ;
         Lval0 = mem_ref(Rval0),
-        Lval  = mem_ref(Rval),
-        Rval  = remap_rval(Remap, Rval0)
+        Rval = remap_rval(Remap, Rval0),
+        Lval = mem_ref(Rval)
     ;
         ( Lval0 = reg(_, _)
         ; Lval0 = succip
@@ -1025,32 +1097,32 @@
 remap_rval(Remap, Rval0) = Rval :-
     (
         Rval0 = lval(Lval0),
-        Rval  = lval(Lval),
-        Lval  = remap_lval(Remap, Lval0)
+        Lval = remap_lval(Remap, Lval0),
+        Rval = lval(Lval)
     ;
         Rval0 = var(_),
         Rval  = Rval0
     ;
         Rval0 = mkword(Tag, Ptr0),
-        Rval  = mkword(Tag, Ptr),
-        Ptr   = remap_rval(Remap, Ptr0)
+        Ptr = remap_rval(Remap, Ptr0),
+        Rval = mkword(Tag, Ptr)
     ;
         Rval0 = const(Const0),
-        Rval  = const(Const),
-        Const = remap_rval_const(Remap, Const0)
+        Const = remap_rval_const(Remap, Const0),
+        Rval = const(Const)
     ;
         Rval0 = unop(Unop, A0),
-        Rval  = unop(Unop, A),
-        A = remap_rval(Remap, A0)
+        A = remap_rval(Remap, A0),
+        Rval = unop(Unop, A)
     ;
         Rval0 = binop(Binop, A0, B0),
-        Rval  = binop(Binop, A, B),
         A = remap_rval(Remap, A0),
-        B = remap_rval(Remap, B0)
+        B = remap_rval(Remap, B0),
+        Rval = binop(Binop, A, B)
     ;
         Rval0 = mem_addr(MemRef0),
-        Rval  = mem_addr(MemRef),
-        MemRef = remap_mem_ref(Remap, MemRef0)
+        MemRef = remap_mem_ref(Remap, MemRef0),
+        Rval = mem_addr(MemRef)
     ).
 
 :- func remap_rval_const(static_cell_remap_info, rval_const) = rval_const.
@@ -1058,18 +1130,17 @@
 remap_rval_const(Remap, Const0) = Const :-
     (
         Const0 = llconst_data_addr(Addr0, MaybeOffset),
-        Const  = llconst_data_addr(Addr,  MaybeOffset),
         (
             Addr0 = data_addr(ModuleName, DataName0),
-            Addr  = data_addr(ModuleName, DataName),
-            DataName = remap_data_name(Remap, DataName0)
-        ;
-            Addr0 = rtti_addr(_),
-            Addr = Addr0
+            DataName = remap_data_name(Remap, DataName0),
+            Addr = data_addr(ModuleName, DataName)
         ;
-            Addr0 = layout_addr(_),
+            ( Addr0 = rtti_addr(_)
+            ; Addr0 = layout_addr(_)
+            ),
             Addr = Addr0
-        )
+        ),
+        Const  = llconst_data_addr(Addr,  MaybeOffset)
     ;
         ( Const0 = llconst_true
         ; Const0 = llconst_false
@@ -1110,15 +1181,14 @@
 
 remap_mem_ref(Remap, MemRef0) = MemRef :-
     (
-        MemRef0 = stackvar_ref(_),
-        MemRef = MemRef0
-    ;
-        MemRef0 = framevar_ref(_),
+        ( MemRef0 = stackvar_ref(_)
+        ; MemRef0 = framevar_ref(_)
+        ),
         MemRef = MemRef0
     ;
         MemRef0 = heap_ref(Ptr0, Tag, FieldNum),
-        MemRef  = heap_ref(Ptr, Tag, FieldNum),
-        Ptr = remap_rval(Remap, Ptr0)
+        Ptr = remap_rval(Remap, Ptr0),
+        MemRef = heap_ref(Ptr, Tag, FieldNum)
     ).
 
 %-----------------------------------------------------------------------------%
Index: compiler/ite_gen.m
===================================================================
RCS file: /home/mercury/mercury1/repository/mercury/compiler/ite_gen.m,v
retrieving revision 1.98
diff -u -b -r1.98 ite_gen.m
--- compiler/ite_gen.m	15 Jan 2007 02:23:46 -0000	1.98
+++ compiler/ite_gen.m	30 Jul 2007 04:28:58 -0000
@@ -25,13 +25,12 @@
 
 %---------------------------------------------------------------------------%
 
-:- pred ite_gen.generate_ite(add_trail_ops::in, code_model::in,
+:- pred generate_ite(code_model::in,
     hlds_goal::in, hlds_goal::in, hlds_goal::in, hlds_goal_info::in,
     code_tree::out, code_info::in, code_info::out) is det.
 
-:- pred ite_gen.generate_negation(add_trail_ops::in, code_model::in,
-    hlds_goal::in, hlds_goal_info::in, code_tree::out,
-    code_info::in, code_info::out) is det.
+:- pred generate_negation(code_model::in, hlds_goal::in, hlds_goal_info::in,
+    code_tree::out, code_info::in, code_info::out) is det.
 
 %---------------------------------------------------------------------------%
 %---------------------------------------------------------------------------%
@@ -41,14 +40,21 @@
 :- import_module backend_libs.builtin_ops.
 :- import_module hlds.goal_form.
 :- import_module hlds.hlds_llds.
+:- import_module hlds.hlds_module.
+:- import_module hlds.hlds_pred.
 :- import_module hlds.instmap.
 :- import_module libs.compiler_util.
 :- import_module libs.globals.
 :- import_module libs.options.
 :- import_module libs.tree.
 :- import_module ll_backend.code_gen.
+:- import_module ll_backend.continuation_info.
 :- import_module ll_backend.trace_gen.
+:- import_module mdbcomp.prim_data.
 :- import_module parse_tree.prog_data.
+:- import_module transform_hlds.
+:- import_module transform_hlds.rbmm.
+:- import_module transform_hlds.rbmm.region_transformation.
 
 :- import_module bool.
 :- import_module int.
@@ -62,8 +68,8 @@
 
 %---------------------------------------------------------------------------%
 
-generate_ite(AddTrailOps, CodeModel, CondGoal0, ThenGoal, ElseGoal,
-        IteGoalInfo, Code, !CI) :-
+generate_ite(CodeModel, CondGoal0, ThenGoal, ElseGoal, IteGoalInfo, Code,
+        !CI) :-
     CondGoal0 = hlds_goal(CondExpr, CondInfo0),
     goal_info_get_code_model(CondInfo0, CondCodeModel),
     (
@@ -77,8 +83,7 @@
 
     goal_info_get_resume_point(CondInfo0, Resume),
     (
-        Resume = resume_point(ResumeVarsPrime, ResumeLocsPrime)
-    ->
+        Resume = resume_point(ResumeVarsPrime, ResumeLocsPrime),
         ResumeVars = ResumeVarsPrime,
         ResumeLocs = ResumeLocsPrime,
         % The pre_goal_update sanity check insists on no_resume_point,
@@ -87,6 +92,7 @@
         goal_info_set_resume_point(no_resume_point, CondInfo0, CondInfo),
         CondGoal = hlds_goal(CondExpr, CondInfo)
     ;
+        Resume = no_resume_point,
         unexpected(this_file,
             "condition of an if-then-else has no resume point")
     ),
@@ -100,9 +106,10 @@
     % This is after code_info.produce_vars since code that
     % flushes the cache may allocate memory we must not "recover".
     code_info.get_globals(!.CI, Globals),
+    globals.lookup_bool_option(Globals, reclaim_heap_on_semidet_failure,
+        ReclaimOption),
     (
-        globals.lookup_bool_option(Globals,
-            reclaim_heap_on_semidet_failure, yes),
+        ReclaimOption = yes,
         goal_may_allocate_heap(CondGoal)
     ->
         ReclaimHeap = yes
@@ -112,28 +119,35 @@
     code_info.maybe_save_hp(ReclaimHeap, SaveHpCode, MaybeHpSlot, !CI),
 
     % Maybe save the current trail state before the condition.
-    % NOTE: this code should be kept up-to-date with the corresponding
+    % NOTE: This code should be kept up-to-date with the corresponding
     %       code for the MLDS backend in add_trail_ops.m.
+    AddTrailOps = should_add_trail_ops(!.CI, IteGoalInfo),
     (
-        AddTrailOps = no,
-        IteTrailOps = no
+        AddTrailOps = do_not_add_trail_ops,
+        IteTrailOps = do_not_add_trail_ops
     ;
-        AddTrailOps = yes,
+        AddTrailOps = add_trail_ops,
         get_opt_trail_ops(!.CI, OptTrailOps),
         (
             OptTrailOps = yes,
             goal_cannot_modify_trail(CondInfo0) = yes,
             CondCodeModel \= model_non
         ->
-            IteTrailOps = no
+            IteTrailOps = do_not_add_trail_ops
         ;
-            IteTrailOps = yes
+            IteTrailOps = add_trail_ops
         )
     ), 
-    
     code_info.maybe_save_ticket(IteTrailOps, SaveTicketCode, MaybeTicketSlot,
         !CI),
 
+    % XXX Consider optimizing IteRegionOps like IteTrailOps.
+    AddRegionOps = should_add_region_ops(!.CI, IteGoalInfo),
+    IteRegionOps = AddRegionOps,
+    goal_to_conj_list(ElseGoal, ElseGoals),
+    maybe_create_ite_region_frame(IteRegionOps, CondInfo, ElseGoals,
+        RegionCondCode, RegionThenCode, RegionElseCode, RegionStackVars, !CI),
+
     code_info.remember_position(!.CI, BranchStart),
 
     code_info.prepare_for_ite_hijack(EffCodeModel, HijackInfo,
@@ -156,18 +170,27 @@
     code_info.make_vars_forward_dead(Zombies, !CI),
 
     % Discard hp and prune trail ticket if the condition succeeded.
-    ( CondCodeModel = model_non ->
+    (
+        CondCodeModel = model_non,
         % We cannot release the stack slots used for the heap pointer
         % and the trail ticket if the condition can be backtracked
         % into.  Nor can we prune the trail ticket that we allocated,
         % since the condition may have allocated other trail tickets
         % since then which have not yet been pruned.
+        %
+        % We also cannot release RegionStackVars.
         code_info.maybe_reset_ticket(MaybeTicketSlot, reset_reason_solve,
             ResetTicketCode)
     ;
+        ( CondCodeModel = model_det
+        ; CondCodeModel = model_semi
+        ),
         code_info.maybe_release_hp(MaybeHpSlot, !CI),
         code_info.maybe_reset_prune_and_release_ticket(MaybeTicketSlot,
-            reset_reason_commit, ResetTicketCode, !CI)
+            reset_reason_commit, ResetTicketCode, !CI),
+
+        code_info.release_several_temp_slots(RegionStackVars,
+            non_persistent_temp_slot, !CI)
     ),
 
     goal_info_get_store_map(IteGoalInfo, StoreMap),
@@ -219,6 +242,7 @@
         FlushCode,
         SaveHpCode,
         SaveTicketCode,
+        RegionCondCode,
         PrepareHijackCode,
         EffectResumeCode,
         CondTraceCode,
@@ -226,6 +250,7 @@
         CondCode,
         ThenNeckCode,
         ResetTicketCode,
+        RegionThenCode,
         ThenTraceCode,
         PNegThenCode,
         ThenCode,
@@ -235,6 +260,7 @@
         ElseNeckCode,
         RestoreHpCode,
         RestoreTicketCode,
+        RegionElseCode,
         ElseTraceCode,
         PNegElseCode,
         ElseCode,
@@ -243,29 +269,34 @@
     code_info.after_all_branches(StoreMap, MaybeEnd, !CI).
 
 %-----------------------------------------------------------------------------%
+%-----------------------------------------------------------------------------%
 
-generate_negation(AddTrailOps, CodeModel, Goal0, NotGoalInfo, Code, !CI) :-
-    ( CodeModel = model_non ->
+generate_negation(CodeModel, Goal0, NotGoalInfo, Code, !CI) :-
+    (
+        CodeModel = model_non,
         unexpected(this_file, "generate_negation: nondet negation.")
     ;
-        true
+        ( CodeModel = model_det
+        ; CodeModel = model_semi
+        )
     ),
 
     Goal0 = hlds_goal(GoalExpr, GoalInfo0),
     goal_info_get_resume_point(GoalInfo0, Resume),
-    ( Resume = resume_point(ResumeVarsPrime, ResumeLocsPrime) ->
+    (
+        Resume = resume_point(ResumeVarsPrime, ResumeLocsPrime),
         ResumeVars = ResumeVarsPrime,
         ResumeLocs = ResumeLocsPrime,
         goal_info_set_resume_point(no_resume_point, GoalInfo0, GoalInfo),
         Goal = hlds_goal(GoalExpr, GoalInfo)
     ;
+        Resume = no_resume_point,
         unexpected(this_file,
             "generate_negation: negated goal has no resume point.")
     ),
-    %
+
     % For a negated simple test, we can generate better code than the general
     % mechanism, because we don't have to flush the cache.
-    %
     (
         CodeModel = model_semi,
         GoalExpr = unify(_, _, _, simple_test(L, R), _),
@@ -293,21 +324,25 @@
                 "test inequality")
         ]),
         code_info.leave_simple_neg(GoalInfo, SimpleNeg, !CI),
-        Code = tree(tree(CodeL, CodeR), TestCode)
+        Code = tree_list([
+            CodeL,
+            CodeR,
+            TestCode
+        ])
     ;
-        generate_negation_general(AddTrailOps, CodeModel, Goal, NotGoalInfo,
+        generate_negation_general(CodeModel, Goal, NotGoalInfo,
             ResumeVars, ResumeLocs, Code, !CI)
     ).
 
     % The code of generate_negation_general is a cut-down version
     % of the code for if-then-elses.
     %
-:- pred generate_negation_general(add_trail_ops::in, code_model::in,
-    hlds_goal::in, hlds_goal_info::in, set(prog_var)::in, resume_locs::in,
-    code_tree::out, code_info::in, code_info::out) is det.
+:- pred generate_negation_general(code_model::in,
+    hlds_goal::in, hlds_goal_info::in, set(prog_var)::in,
+    resume_locs::in, code_tree::out, code_info::in, code_info::out) is det.
 
-generate_negation_general(AddTrailOps, CodeModel, Goal, NotGoalInfo,
-        ResumeVars, ResumeLocs, Code, !CI) :-
+generate_negation_general(CodeModel, Goal, NotGoalInfo, ResumeVars, ResumeLocs,
+        Code, !CI) :-
     code_info.produce_vars(ResumeVars, ResumeMap, FlushCode, !CI),
 
     % Maybe save the heap state current before the condition; this ought to be
@@ -315,9 +350,10 @@
     % get flushed.
 
     code_info.get_globals(!.CI, Globals),
+    globals.lookup_bool_option(Globals, reclaim_heap_on_semidet_failure,
+        ReclaimHeapOnFailure),
     (
-        globals.lookup_bool_option(Globals,
-            reclaim_heap_on_semidet_failure, yes),
+        ReclaimHeapOnFailure = yes,
         goal_may_allocate_heap(Goal)
     ->
         ReclaimHeap = yes
@@ -326,16 +362,25 @@
     ),
     code_info.maybe_save_hp(ReclaimHeap, SaveHpCode, MaybeHpSlot, !CI),
 
-    code_info.maybe_save_ticket(AddTrailOps, SaveTicketCode,
-        MaybeTicketSlot, !CI),
+    % XXX Consider optimizing AddTrailOps as for if-then-elses.
+    AddTrailOps = should_add_trail_ops(!.CI, NotGoalInfo),
+    code_info.maybe_save_ticket(AddTrailOps, SaveTicketCode, MaybeTicketSlot,
+        !CI),
 
-    code_info.prepare_for_ite_hijack(CodeModel, HijackInfo,
-        PrepareHijackCode, !CI),
+    % XXX Consider optimizing IteRegionOps like IteTrailOps.
+    AddRegionOps = should_add_region_ops(!.CI, NotGoalInfo),
+    IteRegionOps = AddRegionOps,
+    Goal = hlds_goal(_, GoalInfo),
+    maybe_create_ite_region_frame(IteRegionOps, GoalInfo, [],
+        RegionCondCode, RegionThenCode, RegionElseCode, RegionStackVars, !CI),
 
-    code_info.make_resume_point(ResumeVars, ResumeLocs, ResumeMap,
-        ResumePoint, !CI),
-    code_info.effect_resume_point(ResumePoint, CodeModel,
-        EffectResumeCode, !CI),
+    code_info.prepare_for_ite_hijack(CodeModel, HijackInfo, PrepareHijackCode,
+        !CI),
+
+    code_info.make_resume_point(ResumeVars, ResumeLocs, ResumeMap, ResumePoint,
+        !CI),
+    code_info.effect_resume_point(ResumePoint, CodeModel, EffectResumeCode,
+        !CI),
 
     % Generate the negated goal as a semi-deterministic goal; it cannot be
     % nondet, since mode correctness requires it to have no output vars.
@@ -350,12 +395,16 @@
 
     code_info.get_forward_live_vars(!.CI, LiveVars),
 
-    ( CodeModel = model_det ->
+    (
+        CodeModel = model_det,
         % The then branch will never be reached.
         PruneTicketCode = empty,
         FailTraceCode = empty,
         FailCode = empty
     ;
+        ( CodeModel = model_semi
+        ; CodeModel = model_non
+        ),
         code_info.remember_position(!.CI, AfterNegatedGoal),
         % The call to reset_ticket(..., commit) here is necessary
         % in order to properly detect floundering.
@@ -380,6 +429,8 @@
     code_info.maybe_restore_and_release_hp(MaybeHpSlot, RestoreHpCode, !CI),
     code_info.maybe_reset_discard_and_release_ticket(MaybeTicketSlot,
         reset_reason_undo, RestoreTicketCode, !CI),
+    code_info.release_several_temp_slots(RegionStackVars,
+        non_persistent_temp_slot, !CI),
     maybe_generate_negated_event_code(Goal, NotGoalInfo, neg_success,
         SuccessTraceCode, !CI),
 
@@ -391,11 +442,13 @@
         EffectResumeCode,
         SaveHpCode,
         SaveTicketCode,
+        RegionCondCode,
         EnterTraceCode,
         PNegCondCode,
         GoalCode,
         ThenNeckCode,
         PruneTicketCode,
+        RegionThenCode,
         FailTraceCode,
         PNegThenCode,
         FailCode,
@@ -403,8 +456,10 @@
         ElseNeckCode,
         RestoreTicketCode,
         RestoreHpCode,
+        RegionElseCode,
         SuccessTraceCode,
-        PNegElseCode]).
+        PNegElseCode
+    ]).
 
 %---------------------------------------------------------------------------%
 
@@ -487,6 +542,225 @@
 
 %-----------------------------------------------------------------------------%
 
+:- pred maybe_create_ite_region_frame(add_region_ops::in,
+    hlds_goal_info::in, list(hlds_goal)::in,
+    code_tree::out, code_tree::out, code_tree::out, list(lval)::out,
+    code_info::in, code_info::out) is det.
+
+maybe_create_ite_region_frame(IteRegionOps, CondGoalInfo, ElseGoals,
+        CondCode, ThenCode, ElseCode, StackVars, !CI) :-
+    (
+        IteRegionOps = do_not_add_region_ops,
+        CondCode = empty,
+        ThenCode = empty,
+        ElseCode = empty,
+        StackVars = []
+    ;
+        IteRegionOps = add_region_ops,
+        code_info.get_forward_live_vars(!.CI, ForwardLiveVars),
+        LiveRegionVars = filter_region_vars(!.CI, ForwardLiveVars),
+
+        goal_info_get_nonlocals(CondGoalInfo, CondNonLocals),
+        CondNonlocalRegionVars = filter_region_vars(!.CI, CondNonLocals),
+
+        % XXX CondRemovedRegionVars should be the set of region vars whose
+        % regions are removed in the condition, and CondAllocRegionVars should
+        % be the set of region vars whose regions may be allocated into in the
+        % condition. However, this information is not yet available in the
+        % HLDS, and there is no simple way here to reconstruct it.
+        % (For example, the set of region vars live after the condition
+        % is *not* sufficient to compute CondRemovedRegionVars, since
+        % a region variable can become dead (in the absence of further
+        % references to the region in this procedure) even if the associated
+        % region hasn't been removed.
+        CondRemovedRegionVars = CondNonlocalRegionVars,
+        CondAllocRegionVars = CondNonlocalRegionVars,
+
+        code_info.get_module_info(!.CI, ModuleInfo),
+        find_regions_removed_at_start_of_else(ElseGoals, ModuleInfo,
+            set.init, RemovedAtStartOfElse),
+
+        % XXX We want to compute UnprotectedRemovedAtStartOfElse as the
+        % intersection of RemovedAtStartOfElse and the set of region variables
+        % whose regions are statically known to be unprotected at this point in
+        % the code. However, in the absence of the required program analysis,
+        % we can't know of any statically unprotected regions.
+        UnprotectedRemovedAtStartOfElse = set.init,
+
+        ProtectRegionVars = set.intersect(LiveRegionVars,
+            CondRemovedRegionVars),
+        SnapshotRegionVars0 = set.intersect(LiveRegionVars,
+            CondAllocRegionVars),
+        ( set.empty(UnprotectedRemovedAtStartOfElse) ->
+            SnapshotRegionVars = SnapshotRegionVars0
+        ;
+            SnapshotRegionVars = set.difference(SnapshotRegionVars0,
+                UnprotectedRemovedAtStartOfElse)
+        ),
+
+        ProtectRegionVarList = set.to_sorted_list(ProtectRegionVars),
+        SnapshotRegionVarList = set.to_sorted_list(SnapshotRegionVars),
+
+        list.length(ProtectRegionVarList, NumProtectRegionVars),
+        list.length(SnapshotRegionVarList, NumSnapshotRegionVars),
+
+        code_info.get_globals(!.CI, Globals),
+        globals.lookup_int_option(Globals, size_region_ite_fixed,
+            FixedSize),
+        globals.lookup_int_option(Globals, size_region_ite_protect,
+            ProtectSize),
+        globals.lookup_int_option(Globals, size_region_ite_snapshot,
+            SnapshotSize),
+        FrameSize = FixedSize
+            + ProtectSize * NumProtectRegionVars
+            + SnapshotSize * NumSnapshotRegionVars,
+
+        Items = list.duplicate(FrameSize, slot_region_ite),
+        acquire_several_temp_slots(Items, non_persistent_temp_slot,
+            StackVars, MainStackId, FirstSlotNum, LastSlotNum, !CI),
+        EmbeddedStackFrame = embedded_stack_frame_id(MainStackId,
+            FirstSlotNum, LastSlotNum),
+        FirstNonFixedAddr =
+            first_nonfixed_embedded_slot_addr(EmbeddedStackFrame, FixedSize),
+        acquire_reg(reg_r, ProtectNumRegLval, !CI),
+        acquire_reg(reg_r, SnapshotNumRegLval, !CI),
+        acquire_reg(reg_r, AddrRegLval, !CI),
+        PushInitCode = node([
+            llds_instr(
+                push_region_frame(region_stack_ite, EmbeddedStackFrame),
+                "Save stack pointer of embedded region ite stack"),
+            llds_instr(
+                assign(ProtectNumRegLval, const(llconst_int(0))),
+                "Initialize number of protect_infos"),
+            llds_instr(
+                assign(SnapshotNumRegLval, const(llconst_int(0))),
+                "Initialize number of snapshot_infos"),
+            llds_instr(
+                assign(AddrRegLval, FirstNonFixedAddr),
+                "Initialize pointer to nonfixed part of embedded frame")
+        ]),
+        ite_protect_regions(ProtectNumRegLval, AddrRegLval,
+            EmbeddedStackFrame, ProtectRegionVarList, ProtectRegionCode, !CI),
+        ite_alloc_snapshot_regions(SnapshotNumRegLval, AddrRegLval,
+            EmbeddedStackFrame, RemovedAtStartOfElse, SnapshotRegionVarList,
+            SnapshotRegionCode, !CI),
+        SetCode = node([
+            llds_instr(
+                region_set_fixed_slot(region_set_ite_num_protects,
+                    EmbeddedStackFrame, lval(ProtectNumRegLval)),
+                "Store the number of protect_infos"),
+            llds_instr(
+                region_set_fixed_slot(region_set_ite_num_snapshots,
+                    EmbeddedStackFrame, lval(SnapshotNumRegLval)),
+                "Store the number of snapshot_infos")
+        ]),
+        release_reg(ProtectNumRegLval, !CI),
+        release_reg(SnapshotNumRegLval, !CI),
+        release_reg(AddrRegLval, !CI),
+
+        goal_info_get_code_model(CondGoalInfo, CondCodeModel),
+        (
+            CondCodeModel = model_non,
+            CondKind = region_ite_nondet_cond
+        ;
+            CondCodeModel = model_semi,
+            CondKind = region_ite_semidet_cond
+        ;
+            CondCodeModel = model_det,
+            unexpected(this_file, "maybe_create_ite_region_frame: det cond")
+        ),
+
+        CondCode = tree_list([
+            PushInitCode,
+            ProtectRegionCode,
+            SnapshotRegionCode,
+            SetCode
+        ]),
+        ThenCode = node([
+            llds_instr(
+                use_and_maybe_pop_region_frame(region_ite_then(CondKind),
+                    EmbeddedStackFrame),
+                "region enter then")
+        ]),
+        ElseCode = node([
+            llds_instr(
+                use_and_maybe_pop_region_frame(region_ite_else(CondKind),
+                    EmbeddedStackFrame),
+                "region enter else")
+        ])
+
+        % XXX A model_non condition can succeed more than once, so the
+        % region_ite_then(region_ite_nondet_cond) operation cannot pop
+        % the ite stack frame. We need to pop this frame when the condition
+        % fails *after* succeeding at least once. This requires modifying the
+        % failure continuation and/or the resume point. This has not yet been
+        % implemented.
+    ).
+
+    % Given the list of goals in the else branch, accumulate the region
+    % variables whose regions are removed at the start of that list.
+    %
+:- pred find_regions_removed_at_start_of_else(list(hlds_goal)::in,
+    module_info::in, set(prog_var)::in, set(prog_var)::out) is det.
+
+find_regions_removed_at_start_of_else([], _, !Removed).
+find_regions_removed_at_start_of_else([Goal | Goals], ModuleInfo, !Removed) :-
+    Goal = hlds_goal(GoalExpr, _),
+    (
+        GoalExpr = plain_call(PredId, _ProcId, Args, _Builtin, _UC, _SymName),
+        module_info_pred_info(ModuleInfo, PredId, PredInfo),
+        pred_info_module(PredInfo) = mercury_region_builtin_module,
+        pred_info_name(PredInfo) = remove_region_pred_name,
+        Args = [RegionVar]
+    ->
+        set.insert(!.Removed, RegionVar, !:Removed),
+        find_regions_removed_at_start_of_else(Goals, ModuleInfo, !Removed)
+    ;
+        true
+    ).
+
+:- pred ite_protect_regions(lval::in, lval::in, embedded_stack_frame_id::in,
+    list(prog_var)::in, code_tree::out, code_info::in, code_info::out) is det.
+
+ite_protect_regions(_, _, _, [], empty, !CI).
+ite_protect_regions(NumLval, AddrLval, EmbeddedStackFrame,
+        [RegionVar | RegionVars], tree(Code, Codes), !CI) :-
+    produce_variable(RegionVar, ProduceVarCode, RegionVarRval, !CI),
+    SaveCode = node([
+        llds_instr(
+            region_fill_frame(region_fill_ite_protect,
+                EmbeddedStackFrame, RegionVarRval, NumLval, AddrLval),
+            "ite protect the region if needed")
+    ]),
+    Code = tree(ProduceVarCode, SaveCode),
+    ite_protect_regions(NumLval, AddrLval, EmbeddedStackFrame,
+        RegionVars, Codes, !CI).
+
+:- pred ite_alloc_snapshot_regions(lval::in, lval::in,
+    embedded_stack_frame_id::in, set(prog_var)::in,
+    list(prog_var)::in, code_tree::out, code_info::in, code_info::out) is det.
+
+ite_alloc_snapshot_regions(_, _, _, _, [], empty, !CI).
+ite_alloc_snapshot_regions(NumLval, AddrLval, EmbeddedStackFrame,
+        RemovedVars, [RegionVar | RegionVars], tree(Code, Codes), !CI) :-
+    produce_variable(RegionVar, ProduceVarCode, RegionVarRval, !CI),
+    ( set.member(RegionVar, RemovedVars) ->
+        RemovedAtStartOfElse = removed_at_start_of_else
+    ;
+        RemovedAtStartOfElse = not_removed_at_start_of_else
+    ),
+    SaveCode = node([
+        llds_instr(
+            region_fill_frame(region_fill_ite_snapshot(RemovedAtStartOfElse),
+                EmbeddedStackFrame, RegionVarRval, NumLval, AddrLval),
+            "take alloc snapshot of the region")
+    ]),
+    Code = tree(ProduceVarCode, SaveCode),
+    ite_alloc_snapshot_regions(NumLval, AddrLval, EmbeddedStackFrame,
+        RemovedVars, RegionVars, Codes, !CI).
+
+%-----------------------------------------------------------------------------%
+
 :- func this_file = string.
 
 this_file = "ite_gen".
Index: compiler/jumpopt.m
===================================================================
RCS file: /home/mercury/mercury1/repository/mercury/compiler/jumpopt.m,v
retrieving revision 1.104
diff -u -b -r1.104 jumpopt.m
--- compiler/jumpopt.m	9 Jul 2007 13:28:31 -0000	1.104
+++ compiler/jumpopt.m	30 Jul 2007 04:53:20 -0000
@@ -692,6 +692,17 @@
             NewRemain = specified(NewInstrs, Instrs0)
         )
     ;
+        Uinstr0 = keep_assign(Lval, Rval0),
+        % Any labels mentioned in Rval0 should be short-circuited.
+        jumpopt.short_labels_rval(Instrmap, Rval0, Rval),
+        ( Rval = Rval0 ->
+            NewRemain = usual_case
+        ;
+            Shorted = Comment0 ++ " (some shortcircuits)",
+            NewInstrs = [llds_instr(keep_assign(Lval, Rval), Shorted)],
+            NewRemain = specified(NewInstrs, Instrs0)
+        )
+    ;
         Uinstr0 = mkframe(FrameInfo, Redoip),
         ( Redoip = yes(code_label(Label0)) ->
             jumpopt.short_label(Instrmap, Label0, Label),
@@ -805,6 +816,10 @@
         ; Uinstr0 = incr_sp(_, _, _)
         ; Uinstr0 = decr_sp(_)
         ; Uinstr0 = decr_sp_and_return(_)
+        ; Uinstr0 = push_region_frame(_, _)
+        ; Uinstr0 = region_fill_frame(_, _, _, _, _)
+        ; Uinstr0 = region_set_fixed_slot(_, _, _)
+        ; Uinstr0 = use_and_maybe_pop_region_frame(_, _)
         ; Uinstr0 = store_ticket(_)
         ; Uinstr0 = reset_ticket(_, _)
         ; Uinstr0 = discard_ticket
Index: compiler/livemap.m
===================================================================
RCS file: /home/mercury/mercury1/repository/mercury/compiler/livemap.m,v
retrieving revision 1.87
diff -u -b -r1.87 livemap.m
--- compiler/livemap.m	9 Jul 2007 13:28:32 -0000	1.87
+++ compiler/livemap.m	30 Jul 2007 05:47:13 -0000
@@ -47,6 +47,7 @@
 :- import_module bool.
 :- import_module pair.
 :- import_module string.
+:- import_module svset.
 
 %-----------------------------------------------------------------------------%
 %-----------------------------------------------------------------------------%
@@ -140,7 +141,9 @@
         Uinstr0 = block(_, _, _),
         unexpected(this_file, "block found in backward scan in build_livemap")
     ;
-        Uinstr0 = assign(Lval, Rval),
+        ( Uinstr0 = assign(Lval, Rval)
+        ; Uinstr0 = keep_assign(Lval, Rval)
+        ),
 
         % Make dead the variable assigned, but make any variables
         % needed to access it live. Make the variables in the assigned
@@ -149,7 +152,7 @@
         % appears on the right hand side as well as the left, then we
         % want make_live to put it back into the liveval set.
 
-        set.delete(!.Livevals, Lval, !:Livevals),
+        svset.delete(Lval, !Livevals),
         opt_util.lval_access_rvals(Lval, Rvals),
         livemap.make_live_in_rvals([Rval | Rvals], !Livevals)
     ;
@@ -223,7 +226,7 @@
         )
     ;
         Uinstr0 = save_maxfr(Lval),
-        set.delete(!.Livevals, Lval, !:Livevals),
+        svset.delete(Lval, !Livevals),
         opt_util.lval_access_rvals(Lval, Rvals),
         livemap.make_live_in_rvals(Rvals, !Livevals)
     ;
@@ -239,7 +242,7 @@
         % to lval, but the two should never have any variables in
         % common. This is why doing the deletion first works.
 
-        set.delete(!.Livevals, Lval, !:Livevals),
+        svset.delete(Lval, !Livevals),
         opt_util.lval_access_rvals(Lval, Rvals0),
         (
             MaybeRegionRval = no,
@@ -251,7 +254,7 @@
         livemap.make_live_in_rvals(Rvals, !Livevals)
     ;
         Uinstr0 = mark_hp(Lval),
-        set.delete(!.Livevals, Lval, !:Livevals),
+        svset.delete(Lval, !Livevals),
         opt_util.lval_access_rvals(Lval, Rvals),
         livemap.make_live_in_rvals(Rvals, !Livevals)
     ;
@@ -261,8 +264,31 @@
         Uinstr0 = free_heap(Rval),
         livemap.make_live_in_rvals([Rval], !Livevals)
     ;
+        Uinstr0 = push_region_frame(_RegionStackId, _EmbeddedStackFrame)
+    ;
+        Uinstr0 = region_fill_frame(_FillOp, _EmbeddedStackFrame, IdRval,
+            NumLval, AddrLval),
+        livemap.make_live_in_rval(IdRval, !Livevals),
+        % The instruction takes the current values in NumLval and AddrLval
+        % as inputs, and then updates those values. This means that they are
+        % live on entry to the instruction, and will stay that way afterward.
+        livemap.make_live_in_rval(lval(NumLval), !Livevals),
+        livemap.make_live_in_rval(lval(AddrLval), !Livevals)
+    ;
+        Uinstr0 = region_set_fixed_slot(_SetOp, _EmbeddedStackFrame,
+            ValueRval),
+        livemap.make_live_in_rval(ValueRval, !Livevals)
+    ;
+        Uinstr0 = use_and_maybe_pop_region_frame(_UseOp, _EmbeddedStackFrame)
+        % XXX We should make all stackvars or framevars in _EmbeddedStackFrame
+        % live, to prevent the compiler from optimizing away assignments to
+        % them. However, at the moment all such assignments are done via
+        % region_fill_frame and region_set_fixed_slot instructions, which
+        % we currently do not ever optimize away, so recording the stack slots
+        % as live would be redundant.
+    ;
         Uinstr0 = store_ticket(Lval),
-        set.delete(!.Livevals, Lval, !:Livevals),
+        svset.delete(Lval, !Livevals),
         opt_util.lval_access_rvals(Lval, Rvals),
         livemap.make_live_in_rvals(Rvals, !Livevals)
     ;
@@ -274,7 +300,7 @@
         Uinstr0 = prune_ticket
     ;
         Uinstr0 = mark_ticket_stack(Lval),
-        set.delete(!.Livevals, Lval, !:Livevals),
+        svset.delete(Lval, !Livevals),
         opt_util.lval_access_rvals(Lval, Rvals),
         livemap.make_live_in_rvals(Rvals, !Livevals)
     ;
Index: compiler/llds.m
===================================================================
RCS file: /home/mercury/mercury1/repository/mercury/compiler/llds.m,v
retrieving revision 1.350
diff -u -b -r1.350 llds.m
--- compiler/llds.m	13 Jul 2007 03:27:26 -0000	1.350
+++ compiler/llds.m	30 Jul 2007 03:53:53 -0000
@@ -255,9 +255,13 @@
             % some local temporary variables.
 
     ;       assign(lval, rval)
-            % assign(Location, Value):
-            % Assign the value specified by rval to the location
-            % specified by lval.
+            % assign(Lval, Rval):
+            % Assign Rval to the location specified by Lval.
+
+    ;       keep_assign(lval, rval)
+            % As assign, but this operation cannot be optimized away or
+            % the target replaced by a temp register, even if the target
+            % *appears* to be unused by the following code.
 
     ;       llcall(code_addr, code_addr, list(liveinfo), term.context,
                 goal_path, call_model)
@@ -346,6 +350,69 @@
             % the top-level cell of the rval is no longer needed. `free' is
             % useless but harmless without conservative garbage collection.
 
+    ;       push_region_frame(region_stack_id, embedded_stack_frame_id)
+            % push_region_frame(RegionStackId, EmbeddedStackId)
+            %
+            % Set the stack pointer of the region stack identified by the
+            % first argument to point to the group of stack slots identified
+            % by the second argument (which specifies to the new top embedded
+            % stack frame on that stack) *after* saving the old value of the
+            % stack pointer in the fixed slot reserved for this purpose in
+            % the new frame.
+            %
+            % The instruction will also fill in whatever other fixed slots
+            % of the new stack frame may be filled in at this time.
+
+    ;       region_fill_frame(region_fill_frame_op, embedded_stack_frame_id,
+                rval, lval, lval)
+            % region_fill_frame(FillOp, EmbeddedStackId,
+            %   RegionId, NumLval, AddrLval)
+            %
+            % EmbeddedStackId should match the parameter of the
+            % push_region_frame instruction that created the embedded stack
+            % frame to which this instruction refers. RegionId should
+            % identify a region (i.e. it should point to the region header).
+            %
+            % If the condition appropriate to FillOp is true, then this
+            % instruction will
+            %
+            % (a) increment NumLval by one, and
+            % (b) store the aspects of the region relevant to FillOp
+            %     in one or more consecutive memory locations starting at
+            %     AddrRval,  after which it will increment AddrRval
+            %     by the number of words this uses.
+            %
+            % If the condition is false, the instruction will do nothing.
+            %
+            % The size of the frame must be big enough that the sequence of
+            % region_fill_frame operations executed on it don't overflow.
+            %
+            % At the end of the sequence, NumLval will be stored back into
+            % a fixed slot in the embedded frame using a region_set_fixed_slot
+            % instruction.
+
+    ;       region_set_fixed_slot(region_set_fixed_op, embedded_stack_frame_id,
+                rval)
+            % region_set_fixed_op(SetOp, EmbeddedStackId, Value)
+            %
+            % Given an embedded stack frame identified by EmbeddedStackId,
+            % set the fixed field of this frame identified by SetOp to Value.
+
+    ;       use_and_maybe_pop_region_frame(region_use_frame_op,
+                embedded_stack_frame_id)
+            % use_and_maybe_pop_region_frame(UseOp, EmbeddedStackId)
+            %
+            % For some values of UseOp, this instruction uses the contents of
+            % the frame identified by EmbeddedStackId (including values saved
+            % by region_set_fixed_op instructions) to operate on the values
+            % recorded in the frame by region_fill_frame instructions.
+            %
+            % For some other values of UseOp, this instruction logically pops
+            % the embedded stack off its stack. (The Mercury stacks are
+            % untouched.)
+            %
+            % For yet other values of UseOp, it does both.
+
     ;       store_ticket(lval)
             % Allocate a new "ticket" and store it in the lval.
             %
@@ -662,6 +729,14 @@
                                 % declarations that the C type name came from.
             ).
 
+:- type add_trail_ops
+    --->    add_trail_ops
+    ;       do_not_add_trail_ops.
+
+:- type add_region_ops
+    --->    add_region_ops
+    ;       do_not_add_region_ops.
+
     % See runtime/mercury_trail.h.
 :- type reset_trail_reason
     --->    reset_reason_undo
@@ -671,7 +746,65 @@
     ;       reset_reason_retry
     ;       reset_reason_gc.
 
-:- type add_trail_ops == bool.
+    % See runtime/mercury_region.h
+    % XXX The documentation is not there yet, but should be there soon.
+:- type region_stack_id
+    --->    region_stack_ite
+    ;       region_stack_disj
+    ;       region_stack_commit.
+
+:- type region_fill_frame_op
+    --->    region_fill_ite_protect
+    ;       region_fill_ite_snapshot(removed_at_start_of_else)
+    ;       region_fill_disj_protect
+    ;       region_fill_disj_snapshot
+    ;       region_fill_commit.
+
+:- type removed_at_start_of_else
+    --->    removed_at_start_of_else
+    ;       not_removed_at_start_of_else.
+
+:- type region_set_fixed_op
+    --->    region_set_ite_num_protects
+    ;       region_set_ite_num_snapshots
+    ;       region_set_disj_num_protects
+    ;       region_set_disj_num_snapshots
+    ;       region_set_commit_num_entries.
+
+:- type region_use_frame_op
+    --->    region_ite_then(region_ite_kind)    % uses; pop only if semi
+    ;       region_ite_else(region_ite_kind)    % uses and pops
+    ;       region_ite_nondet_cond_fail         % pops
+    ;       region_disj_later                   % uses
+    ;       region_disj_last                    % uses and pops
+    ;       region_commit_success               % uses and pops
+    ;       region_commit_failure.              % only pops
+
+:- type region_ite_kind
+    --->    region_ite_semidet_cond
+    ;       region_ite_nondet_cond.
+
+:- type embedded_stack_frame_id
+    --->    embedded_stack_frame_id(
+                % The emdedded stack frame consists of the lvals
+                %
+                %   stack_slot_num_to_lval(StackId, FirstSlot)
+                % to
+                %   stack_slot_num_to_lval(StackId, LastSlot)
+                %
+                % with FirstSlot < LastSlot.
+
+                main_stack,                     % StackId
+                int,                            % FirstSlot
+                int                             % LastSlot
+            ).
+
+    % first_nonfixed_embedded_slot_addr(EmbeddedStackId, FixedSize):
+    %
+    % Return the address of the lowest-address non-fixed slot in the given
+    % embedded stack frame.
+    %
+:- func first_nonfixed_embedded_slot_addr(embedded_stack_frame_id, int) = rval.
 
     % Each call instruction has a list of liveinfo, which stores information
     % about which variables are live after the call (that is, on return).
@@ -679,21 +812,23 @@
     %
 :- type liveinfo
     --->    live_lvalue(
-                layout_locn,
                     % What location does this lifeinfo structure refer to?
-                live_value_type,
+                layout_locn,
+
                     % What is the type of this live value?
-                map(tvar, set(layout_locn))
-                    % For each tvar that is a parameter of the type of this
-                    % value, give the set of locations where the type_info
-                    % variable describing the actual type bound to the
-                    % type parameter may be found.
+                live_value_type,
+
+                % For each tvar that is a parameter of the type of this value,
+                % give the set of locations where the type_info variable
+                % describing the actual type bound to the type parameter
+                % may be found.
                     %
                     % We record all the locations of the typeinfo, in case
-                    % different paths of arriving a this program point
-                    % leave the typeinfo in different sets of locations.
-                    % However, there must be at least type_info location
-                    % that is valid along all paths leading to this point.
+                % different paths of arriving a this program point leave
+                % the typeinfo in different sets of locations. However,
+                % there must be at least type_info location that is valid
+                % along all paths leading to this point.
+                map(tvar, set(layout_locn))
             ).
 
     % For an explanation of this type, see the comment on
@@ -715,6 +850,9 @@
     ;       live_value_hp                  % A stored heap pointer.
     ;       live_value_trail_ptr           % A stored trail pointer.
     ;       live_value_ticket              % A stored ticket.
+    ;       live_value_region_ite
+    ;       live_value_region_disj
+    ;       live_value_region_commit
 
     ;       live_value_var(prog_var, string, mer_type, llds_inst)
             % A variable (the var number and name are for execution tracing;
@@ -756,7 +894,26 @@
 
 :- func key_abs_locn_to_lval(_, abs_locn) = lval.
 
-:- func stack_slot_num_to_lval(code_model, int) = lval.
+:- type main_stack
+    --->    det_stack
+    ;       nondet_stack.
+
+    % Return the id of the stack on which procedures with the given code model
+    % have their stack frames.
+    %
+:- func code_model_to_main_stack(code_model) = main_stack.
+
+    % stack_slot_num_to_lval(StackId, N):
+    %
+    % Return an lval for slot N in a stack frame on StackId.
+    %
+:- func stack_slot_num_to_lval(main_stack, int) = lval.
+
+    % stack_slot_num_to_lval(StackId, N):
+    %
+    % Return an rval for the address of slot N in a stack frame on StackId.
+    %
+:- func stack_slot_num_to_lval_ref(main_stack, int) = rval.
 
     % An lval represents a data location or register that can be used
     % as the target of an assignment.
@@ -826,6 +983,11 @@
             % code address to jump to on successful exit from this nondet
             % procedure.
 
+    ;       succfr_slot(rval)
+            % The succfr slot of the specified nondet stack frame; holds the
+            % address of caller's nondet stack frame.  On successful exit
+            % from this nondet procedure, we will set curfr to this value.
+
     ;       redoip_slot(rval)
             % The redoip slot of the specified nondet stack frame; holds the
             % code address to jump to on failure.
@@ -835,11 +997,6 @@
             % address of the frame that the curfr register should be set to
             % when backtracking through the redoip slot.
 
-    ;       succfr_slot(rval)
-            % The succfr slot of the specified nondet stack frame; holds the
-            % address of caller's nondet stack frame.  On successful exit
-            % from this nondet procedure, we will set curfr to this value.
-
     ;       prevfr_slot(rval)
             % The prevfr slot of the specified nondet stack frame; holds the
             % address of the previous frame on the nondet stack.
@@ -1085,6 +1242,8 @@
             % i.e., something whose size is a word but which may be either
             % signed or unsigned (used for registers, stack slots, etc).
 
+:- func region_stack_id_to_string(region_stack_id) = string.
+
 :- pred break_up_local_label(label::in, proc_label::out, int::out) is det.
 
     % Given a non-var rval, figure out its type.
@@ -1126,8 +1285,19 @@
 
 :- import_module libs.compiler_util.
 
+:- import_module int.
+
 %-----------------------------------------------------------------------------%
 
+first_nonfixed_embedded_slot_addr(EmbeddedStackId, FixedSize) = Rval :-
+    EmbeddedStackId = embedded_stack_frame_id(MainStackId,
+        _FirstSlot, LastSlot),
+    % LastSlot has the lowest address; FirstSlot has the highest address.
+    % The fixed slots are at the lowest addresses.
+    % XXX Quan: we may need a +1 here.
+    LowestAddrNonfixedSlot = LastSlot - FixedSize,
+    Rval = stack_slot_num_to_lval_ref(MainStackId, LowestAddrNonfixedSlot).
+
 stack_slot_to_lval(det_slot(N)) = stackvar(N).
 stack_slot_to_lval(parent_det_slot(N)) = parent_stackvar(N).
 stack_slot_to_lval(nondet_slot(N)) = framevar(N).
@@ -1152,12 +1322,21 @@
 key_abs_locn_to_lval(_, AbsLocn) =
     abs_locn_to_lval(AbsLocn).
 
-stack_slot_num_to_lval(CodeModel, SlotNum) =
-    ( CodeModel = model_non ->
-        framevar(SlotNum)
-    ;
-        stackvar(SlotNum)
-    ).
+code_model_to_main_stack(model_det) = det_stack.
+code_model_to_main_stack(model_semi) = det_stack.
+code_model_to_main_stack(model_non) = nondet_stack.
+
+stack_slot_num_to_lval(det_stack, SlotNum) = stackvar(SlotNum).
+stack_slot_num_to_lval(nondet_stack, SlotNum) = framevar(SlotNum).
+
+stack_slot_num_to_lval_ref(det_stack, SlotNum) =
+    mem_addr(stackvar_ref(const(llconst_int(SlotNum)))).
+stack_slot_num_to_lval_ref(nondet_stack, SlotNum) =
+    mem_addr(framevar_ref(const(llconst_int(SlotNum)))).
+
+region_stack_id_to_string(region_stack_ite) = "region_ite_stack".
+region_stack_id_to_string(region_stack_disj) = "region_disj_stack".
+region_stack_id_to_string(region_stack_commit) = "region_commit_stack".
 
 break_up_local_label(Label, ProcLabel, LabelNum) :-
     (
Index: compiler/llds_out.m
===================================================================
RCS file: /home/mercury/mercury1/repository/mercury/compiler/llds_out.m,v
retrieving revision 1.313
diff -u -b -r1.313 llds_out.m
--- compiler/llds_out.m	27 Jul 2007 04:53:59 -0000	1.313
+++ compiler/llds_out.m	31 Jul 2007 01:41:34 -0000
@@ -1893,6 +1893,9 @@
 output_instr_decls(_, assign(Lval, Rval), !DeclSet, !IO) :-
     output_lval_decls(Lval, !DeclSet, !IO),
     output_rval_decls(Rval, !DeclSet, !IO).
+output_instr_decls(_, keep_assign(Lval, Rval), !DeclSet, !IO) :-
+    output_lval_decls(Lval, !DeclSet, !IO),
+    output_rval_decls(Rval, !DeclSet, !IO).
 output_instr_decls(_, llcall(Target, ContLabel, _, _, _, _), !DeclSet, !IO) :-
     output_code_addr_decls(Target, !DeclSet, !IO),
     output_code_addr_decls(ContLabel, !DeclSet, !IO).
@@ -1962,6 +1965,18 @@
     output_rval_decls(Rval, !DeclSet, !IO).
 output_instr_decls(_, free_heap(Rval), !DeclSet, !IO) :-
     output_rval_decls(Rval, !DeclSet, !IO).
+output_instr_decls(_, push_region_frame(_StackId, _EmbeddedFrame),
+        !DeclSet, !IO).
+output_instr_decls(_, region_fill_frame(_FillOp, _EmbeddedFrame, IdRval,
+        NumLval, AddrLval), !DeclSet, !IO) :-
+    output_rval_decls(IdRval, !DeclSet, !IO),
+    output_lval_decls(NumLval, !DeclSet, !IO),
+    output_lval_decls(AddrLval, !DeclSet, !IO).
+output_instr_decls(_, region_set_fixed_slot(_SetOp, _EmbeddedFrame, ValueRval),
+        !DeclSet, !IO) :-
+    output_rval_decls(ValueRval, !DeclSet, !IO).
+output_instr_decls(_, use_and_maybe_pop_region_frame(_UseOp, _EmbeddedFrame),
+        !DeclSet, !IO).
 output_instr_decls(_, store_ticket(Lval), !DeclSet, !IO) :-
     output_lval_decls(Lval, !DeclSet, !IO).
 output_instr_decls(_, reset_ticket(Rval, _Reason), !DeclSet, !IO) :-
@@ -2259,6 +2274,13 @@
     output_rval_as_type(Rval, Type, !IO),
     io.write_string(";\n", !IO).
 
+output_instruction(keep_assign(Lval, Rval), _, !IO) :-
+    io.write_string("\t", !IO),
+    output_lval_for_assign(Lval, Type, !IO),
+    io.write_string(" = ", !IO),
+    output_rval_as_type(Rval, Type, !IO),
+    io.write_string(";\n", !IO).
+
 output_instruction(llcall(Target, ContLabel, LiveVals, _, _, _), ProfInfo,
         !IO) :-
     ProfInfo = CallerLabel - _,
@@ -2531,6 +2553,119 @@
     output_rval_as_type(Rval, data_ptr, !IO),
     io.write_string(");\n", !IO).
 
+output_instruction(push_region_frame(StackId, EmbeddedFrame), _, !IO) :-
+    (
+        StackId = region_stack_ite,
+        io.write_string("\tMR_push_region_ite_frame", !IO)
+    ;
+        StackId = region_stack_disj,
+        io.write_string("\tMR_push_region_disj_frame", !IO)
+    ;
+        StackId = region_stack_commit,
+        io.write_string("\tMR_push_region_commit_frame", !IO)
+    ),
+    io.write_string("(", !IO),
+    output_embedded_frame_addr(EmbeddedFrame, !IO),
+    io.write_string(");", !IO),
+
+    % The comment is to make the code easier to debug;
+    % we can stop printing it out once that has been done.
+    EmbeddedFrame = embedded_stack_frame_id(_StackId, FirstSlot, LastSlot),
+    Comment = " /* " ++ int_to_string(FirstSlot) ++ ".." ++
+        int_to_string(LastSlot) ++ " */",
+    io.write_string(Comment, !IO),
+
+    io.write_string("\n", !IO).
+
+output_instruction(region_fill_frame(FillOp, EmbeddedFrame, IdRval,
+        NumLval, AddrLval), _, !IO) :-
+    (
+        FillOp = region_fill_ite_protect,
+        io.write_string("\tMR_region_fill_ite_protect", !IO)
+    ;
+        FillOp = region_fill_ite_snapshot(removed_at_start_of_else),
+        io.write_string("\tMR_region_fill_ite_snapshot_removed", !IO)
+    ;
+        FillOp = region_fill_ite_snapshot(not_removed_at_start_of_else),
+        io.write_string("\tMR_region_fill_ite_snapshot_not_removed", !IO)
+    ;
+        FillOp = region_fill_disj_protect,
+        io.write_string("\tMR_region_fill_disj_protect", !IO)
+    ;
+        FillOp = region_fill_disj_snapshot,
+        io.write_string("\tMR_region_fill_disj_snapshot", !IO)
+    ;
+        FillOp = region_fill_commit,
+        io.write_string("\tMR_region_fill_commit", !IO)
+    ),
+    io.write_string("(", !IO),
+    output_embedded_frame_addr(EmbeddedFrame, !IO),
+    io.write_string(", ", !IO),
+    output_rval(IdRval, !IO),
+    io.write_string(", ", !IO),
+    output_lval(NumLval, !IO),
+    io.write_string(", ", !IO),
+    output_lval(AddrLval, !IO),
+    io.write_string(");\n", !IO).
+
+output_instruction(region_set_fixed_slot(SetOp, EmbeddedFrame, ValueRval),
+        _, !IO) :-
+    (
+        SetOp = region_set_ite_num_protects,
+        io.write_string("\tMR_region_set_ite_num_protects", !IO)
+    ;
+        SetOp = region_set_ite_num_snapshots,
+        io.write_string("\tMR_region_set_ite_num_snapshots", !IO)
+    ;
+        SetOp = region_set_disj_num_protects,
+        io.write_string("\tMR_region_set_disj_num_protects", !IO)
+    ;
+        SetOp = region_set_disj_num_snapshots,
+        io.write_string("\tMR_region_set_disj_num_snapshots", !IO)
+    ;
+        SetOp = region_set_commit_num_entries,
+        io.write_string("\tMR_region_set_commit_num_entries", !IO)
+    ),
+    io.write_string("(", !IO),
+    output_embedded_frame_addr(EmbeddedFrame, !IO),
+    io.write_string(", ", !IO),
+    output_rval(ValueRval, !IO),
+    io.write_string(");\n", !IO).
+
+output_instruction(use_and_maybe_pop_region_frame(UseOp, EmbeddedFrame),
+        _, !IO) :-
+    (
+        UseOp = region_ite_then(region_ite_semidet_cond),
+        io.write_string("\tMR_use_region_ite_then_semidet", !IO)
+    ;
+        UseOp = region_ite_then(region_ite_nondet_cond),
+        io.write_string("\tMR_use_region_ite_then_nondet", !IO)
+    ;
+        UseOp = region_ite_else(region_ite_semidet_cond),
+        io.write_string("\tMR_use_region_ite_else_semidet", !IO)
+    ;
+        UseOp = region_ite_else(region_ite_nondet_cond),
+        io.write_string("\tMR_use_region_ite_else_nondet", !IO)
+    ;
+        UseOp = region_ite_nondet_cond_fail,
+        io.write_string("\tMR_use_region_ite_nondet_cond_fail", !IO)
+    ;
+        UseOp = region_disj_later,
+        io.write_string("\tMR_use_region_disj_later", !IO)
+    ;
+        UseOp = region_disj_last,
+        io.write_string("\tMR_use_region_disj_last", !IO)
+    ;
+        UseOp = region_commit_success,
+        io.write_string("\tMR_use_region_commit_success", !IO)
+    ;
+        UseOp = region_commit_failure,
+        io.write_string("\tMR_use_region_commit_failure", !IO)
+    ),
+    io.write_string("(", !IO),
+    output_embedded_frame_addr(EmbeddedFrame, !IO),
+    io.write_string(");\n", !IO).
+
 output_instruction(store_ticket(Lval), _, !IO) :-
     io.write_string("\tMR_store_ticket(", !IO),
     output_lval_as_word(Lval, !IO),
@@ -2617,6 +2752,23 @@
     output_label_as_code_addr(Label, !IO),
     io.write_string(");\n", !IO).
 
+    % Our stacks grow upwards in that new stack frames have higher addresses
+    % than old stack frames, but within in each stack frame, we compute the
+    % address of stackvar N or framevar N by *subtracting* N from the address
+    % of the top of (the non-fixed part of) the stack frame, so that e.g.
+    % framevar N+1 is actually stored at a *lower* address than framevar N.
+    %
+    % The C code we interact with refers to embedded stack frames by the
+    % starting (i.e. lowest) address.
+    %
+:- pred output_embedded_frame_addr(embedded_stack_frame_id::in,
+    io::di, io::uo) is det.
+
+output_embedded_frame_addr(EmbeddedFrame, !IO) :-
+    EmbeddedFrame = embedded_stack_frame_id(MainStackId, _FirstSlot, LastSlot),
+    FrameStartRval = stack_slot_num_to_lval_ref(MainStackId, LastSlot),
+    output_rval_as_type(FrameStartRval, data_ptr, !IO).
+
 :- func max_leaf_stack_frame_size = int.
 
 % This should be kept in sync with the value of MR_stack_margin_size
@@ -2988,6 +3140,12 @@
     io.write_string("type trail_ptr", !IO).
 output_live_value_type(live_value_ticket, !IO) :-
     io.write_string("type ticket", !IO).
+output_live_value_type(live_value_region_disj, !IO) :-
+    io.write_string("type region disj", !IO).
+output_live_value_type(live_value_region_commit, !IO) :-
+    io.write_string("type region commit", !IO).
+output_live_value_type(live_value_region_ite, !IO) :-
+    io.write_string("type region ite", !IO).
 output_live_value_type(live_value_unwanted, !IO) :-
     io.write_string("unwanted", !IO).
 output_live_value_type(live_value_var(Var, Name, Type, LldsInst), !IO) :-
Index: compiler/llds_to_x86_64.m
===================================================================
RCS file: /home/mercury/mercury1/repository/mercury/compiler/llds_to_x86_64.m,v
retrieving revision 1.6
diff -u -b -r1.6 llds_to_x86_64.m
--- compiler/llds_to_x86_64.m	17 Jul 2007 05:31:50 -0000	1.6
+++ compiler/llds_to_x86_64.m	30 Jul 2007 04:12:24 -0000
@@ -192,7 +192,10 @@
     transform_livevals(!.RegMap, List, Instrs).
 instr_to_x86_64(!RegMap, block(_, _, CInstrs), Instrs) :-
     transform_block_instr(!.RegMap, CInstrs, Instrs).
-instr_to_x86_64(!RegMap, assign(Lval, Rval), Instrs) :-
+instr_to_x86_64(!RegMap, Op, Instrs) :-
+    ( Op = assign(Lval, Rval)
+    ; Op = keep_assign(Lval, Rval)
+    ),
     transform_lval(!RegMap, Lval, Res0, Res1),
     transform_rval(!RegMap, Rval, Res2, Res3),
     (
@@ -381,11 +384,20 @@
     SetTag = x86_64_instr(or(operand_imm(imm32(int32(Tag))), TempReg2)),
     Instr1 = x86_64_instr(mov(TempReg2, LvalOp)),
     Instrs = IncrAddrInstrs ++ [ImmToReg] ++ [SetTag] ++ [Instr1].
-instr_to_x86_64(!RegMap, mark_hp(_), [x86_64_comment("<<mark_hp>>")]).
+instr_to_x86_64(!RegMap, mark_hp(_), Instr) :-
+    Instr = [x86_64_comment("<<mark_hp>>")].
 instr_to_x86_64(!RegMap, restore_hp(_), Instr) :-
     Instr = [x86_64_comment("<<restore_hp>>")].
 instr_to_x86_64(!RegMap, free_heap(_), Instr) :-
     Instr = [x86_64_comment("<<free_heap>>")].
+instr_to_x86_64(!RegMap, push_region_frame(_, _), Instr) :-
+    Instr = [x86_64_comment("<<push_region_frame>>")].
+instr_to_x86_64(!RegMap, region_fill_frame(_, _, _, _, _), Instr) :-
+    Instr = [x86_64_comment("<<region_fill_frame>>")].
+instr_to_x86_64(!RegMap, region_set_fixed_slot(_, _, _), Instr) :-
+    Instr = [x86_64_comment("<<region_set_fixed_slot>>")].
+instr_to_x86_64(!RegMap, use_and_maybe_pop_region_frame(_, _), Instr) :-
+    Instr = [x86_64_comment("<<use_and_maybe_pop_region_frame>>")].
 instr_to_x86_64(!RegMap, store_ticket(_), Instr) :-
     Instr = [x86_64_comment("<<store_ticket>>")].
 instr_to_x86_64(!RegMap, reset_ticket(_, _), Instr) :-
Index: compiler/lookup_switch.m
===================================================================
RCS file: /home/mercury/mercury1/repository/mercury/compiler/lookup_switch.m,v
retrieving revision 1.75
diff -u -b -r1.75 lookup_switch.m
--- compiler/lookup_switch.m	13 Jul 2007 03:27:27 -0000	1.75
+++ compiler/lookup_switch.m	23 Jul 2007 15:15:47 -0000
@@ -107,9 +107,9 @@
                 assoc_list(int, soln_consts),
                 set(prog_var),          % The resume vars.
                 bool                    % The Boolean "or" of the result
-                                        % of invoking should_emit_trail_ops
-                                        % on the goal_infos of the
-                                        % disjunctions.
+                                        % of invoking goal_may_modify_trail
+                                        % on the goal_infos of the switch arms
+                                        % that are disjunctions.
             ).
 
 :- type soln_consts
@@ -242,7 +242,7 @@
     code_info.remember_position(!.CI, CurPos),
     generate_constants_for_lookup_switch(TaggedCases, OutVars, StoreMap,
         CaseSolns, !MaybeEnd, MaybeLiveness, set.init, ResumeVars,
-        no, GoalTrailOps, !CI),
+        no, GoalsMayModifyTrail, !CI),
     code_info.reset_to_position(CurPos, !CI),
     (
         MaybeLiveness = yes(Liveness)
@@ -257,7 +257,8 @@
         CaseConsts = all_one_soln(CaseValuePairs),
         assoc_list.values(CaseValuePairs, CaseValues)
     ;
-        CaseConsts = some_several_solns(CaseSolns, ResumeVars, GoalTrailOps),
+        CaseConsts = some_several_solns(CaseSolns, ResumeVars,
+            GoalsMayModifyTrail),
         % This generates CaseValues in reverse order of index, but given that
         % we only use CaseValues to find out the right LLDSTypes, this is OK.
         project_solns_to_rval_lists(CaseSolns, [], CaseValues)
@@ -321,7 +322,7 @@
         !ResumeVars, !GoalTrailOps, !CI).
 generate_constants_for_lookup_switch([Case | Cases], Vars, StoreMap,
         [CaseVal | Rest], !MaybeEnd, MaybeLiveness, !ResumeVars,
-        !GoalTrailOps, !CI) :-
+        !GoalsMayModifyTrail, !CI) :-
     Case = extended_case(_, int_tag(CaseTag), _, Goal),
     Goal = hlds_goal(GoalExpr, GoalInfo),
 
@@ -331,7 +332,7 @@
     not set.member(feature_save_deep_excp_vars, Features),
 
     ( GoalExpr = disj(Disjuncts) ->
-        bool.or(goal_may_modify_trail(GoalInfo), !GoalTrailOps),
+        bool.or(goal_may_modify_trail(GoalInfo), !GoalsMayModifyTrail),
         (
             Disjuncts = [],
             % Cases like this should have been filtered out by
@@ -372,7 +373,7 @@
         CaseVal = CaseTag - one_soln(Soln)
     ),
     generate_constants_for_lookup_switch(Cases, Vars, StoreMap, Rest,
-        !MaybeEnd, _, !ResumeVars, !GoalTrailOps, !CI).
+        !MaybeEnd, _, !ResumeVars, !GoalsMayModifyTrail, !CI).
 
 %---------------------------------------------------------------------------%
 
@@ -412,9 +413,16 @@
             StartVal, EndVal, CaseValues, OutVars, LLDSTypes,
             NeedBitVecCheck, Liveness, RestCode, !CI)
     ;
-        CaseConsts = some_several_solns(CaseSolns, ResumeVars, GoalTrailOps),
+        CaseConsts = some_several_solns(CaseSolns, ResumeVars,
+            GoalsMayModifyTrail),
         get_emit_trail_ops(!.CI, EmitTrailOps),
-        bool.and(EmitTrailOps, GoalTrailOps, AddTrailOps),
+        (
+            GoalsMayModifyTrail = yes,
+            AddTrailOps = EmitTrailOps
+        ;
+            GoalsMayModifyTrail = no,
+            AddTrailOps = do_not_add_trail_ops
+        ),
         Comment = node([
             llds_instr(comment("several soln lookup switch"), "")
         ]),
@@ -525,9 +533,9 @@
 
 :- pred generate_several_soln_lookup_switch(rval::in, abs_store_map::in,
     branch_end::in, int::in, int::in, assoc_list(int, soln_consts)::in,
-    set(prog_var)::in, bool::in, list(prog_var)::in, list(llds_type)::in,
-    need_bit_vec_check::in, set(prog_var)::in, code_tree::out,
-    code_info::in, code_info::out) is det.
+    set(prog_var)::in, add_trail_ops::in, list(prog_var)::in,
+    list(llds_type)::in, need_bit_vec_check::in, set(prog_var)::in,
+    code_tree::out, code_info::in, code_info::out) is det.
 
 generate_several_soln_lookup_switch(IndexRval, StoreMap, MaybeEnd0,
         StartVal, EndVal, CaseSolns, ResumeVars, AddTrailOps, OutVars,
@@ -588,10 +596,10 @@
     % We cannot release the stack slots anywhere, since they will be needed
     % after backtracking to later alternatives of any model_non switch arm.
     code_info.acquire_reg_not_in_storemap(StoreMap, BaseReg, !CI),
-    code_info.acquire_temp_slot(lookup_switch_cur, non_persistent_temp_slot,
-        CurSlot, !CI),
-    code_info.acquire_temp_slot(lookup_switch_max, non_persistent_temp_slot,
-        MaxSlot, !CI),
+    code_info.acquire_temp_slot(slot_lookup_switch_cur,
+        non_persistent_temp_slot, CurSlot, !CI),
+    code_info.acquire_temp_slot(slot_lookup_switch_max,
+        non_persistent_temp_slot, MaxSlot, !CI),
     % IndexRval has already had Start subtracted from it.
     BaseRval = binop(int_mul, IndexRval, const(llconst_int(MainRowWidth))),
     BaseRegInitCode = node([
@@ -632,8 +640,8 @@
 
 :- pred generate_code_for_each_kind(assoc_list(int, case_kind)::in,
     lval::in, lval::in, lval::in, rval::in, label::in, position_info::in,
-    set(prog_var)::in, bool::in, list(prog_var)::in, abs_store_map::in,
-    branch_end::in, set(prog_var)::in, code_tree::out,
+    set(prog_var)::in, add_trail_ops::in, list(prog_var)::in,
+    abs_store_map::in, branch_end::in, set(prog_var)::in, code_tree::out,
     code_info::in, code_info::out) is det.
 
 generate_code_for_each_kind([], _, _, _, _, _, _, _, _, _, _, _, _, _, !CI) :-
Index: compiler/middle_rec.m
===================================================================
RCS file: /home/mercury/mercury1/repository/mercury/compiler/middle_rec.m,v
retrieving revision 1.129
diff -u -b -r1.129 middle_rec.m
--- compiler/middle_rec.m	9 Jul 2007 13:28:32 -0000	1.129
+++ compiler/middle_rec.m	30 Jul 2007 04:55:52 -0000
@@ -509,6 +509,9 @@
 find_used_registers_instr(assign(Lval, Rval), !Used) :-
     find_used_registers_lval(Lval, !Used),
     find_used_registers_rval(Rval, !Used).
+find_used_registers_instr(keep_assign(Lval, Rval), !Used) :-
+    find_used_registers_lval(Lval, !Used),
+    find_used_registers_rval(Rval, !Used).
 find_used_registers_instr(llcall(_, _, _, _, _, _), !Used).
 find_used_registers_instr(mkframe(_, _), !Used).
 find_used_registers_instr(label(_), !Used).
@@ -538,6 +541,17 @@
     find_used_registers_rval(Rval, !Used).
 find_used_registers_instr(free_heap(Rval), !Used) :-
     find_used_registers_rval(Rval, !Used).
+find_used_registers_instr(push_region_frame(_Id, _EmbeddedStackFrame), !Used).
+find_used_registers_instr(region_fill_frame(_FillOp, _EmbeddedStackFrame,
+        IdRval, NumLval, AddrLval), !Used) :-
+    find_used_registers_rval(IdRval, !Used),
+    find_used_registers_lval(NumLval, !Used),
+    find_used_registers_lval(AddrLval, !Used).
+find_used_registers_instr(region_set_fixed_slot(_SetOp, _EmbeddedStackFrame,
+        ValueRval), !Used) :-
+    find_used_registers_rval(ValueRval, !Used).
+find_used_registers_instr(use_and_maybe_pop_region_frame(_UseOp,
+    _EmbeddedStackFrame), !Used).
 find_used_registers_instr(store_ticket(Lval), !Used) :-
     find_used_registers_lval(Lval, !Used).
 find_used_registers_instr(reset_ticket(Rval, _Rsn), !Used) :-
Index: compiler/opt_debug.m
===================================================================
RCS file: /home/mercury/mercury1/repository/mercury/compiler/opt_debug.m,v
retrieving revision 1.195
diff -u -b -r1.195 opt_debug.m
--- compiler/opt_debug.m	17 Jul 2007 23:48:28 -0000	1.195
+++ compiler/opt_debug.m	30 Jul 2007 05:02:07 -0000
@@ -649,6 +649,10 @@
         Str = dump_lval(yes(ProcLabel), Lval) ++ " := " ++
             dump_rval(yes(ProcLabel), Rval)
     ;
+        Instr = keep_assign(Lval, Rval),
+        Str = "keep " ++ dump_lval(yes(ProcLabel), Lval) ++ " := " ++
+            dump_rval(yes(ProcLabel), Rval)
+    ;
         Instr = llcall(Callee, ReturnLabel, _LiveInfo, _Context, _GoalPath,
             CallModel),
         (
@@ -762,6 +766,94 @@
         Instr = free_heap(Rval),
         Str = "free_heap(" ++ dump_rval(yes(ProcLabel), Rval) ++ ")"
     ;
+        Instr = push_region_frame(StackId, EmbeddedStackFrame),
+        Str = "push_region_frame(" ++
+            region_stack_id_to_string(StackId) ++ "," ++
+            dump_embedded_stack_frame_id(EmbeddedStackFrame) ++ ")"
+    ;
+        Instr = region_fill_frame(FillOp, EmbeddedStackFrame, IdRval,
+            NumLval, AddrLval),
+        (
+            FillOp = region_fill_ite_protect,
+            FillOpStr = "ite_protect"
+        ;
+            FillOp = region_fill_ite_snapshot(removed_at_start_of_else),
+            FillOpStr = "ite_snapshot(removed_at_start_of_else)"
+        ;
+            FillOp = region_fill_ite_snapshot(not_removed_at_start_of_else),
+            FillOpStr = "ite_snapshot(not_removed_at_start_of_else)"
+        ;
+            FillOp = region_fill_disj_protect,
+            FillOpStr = "disj_protect"
+        ;
+            FillOp = region_fill_disj_snapshot,
+            FillOpStr = "disj_snapshot"
+        ;
+            FillOp = region_fill_commit,
+            FillOpStr = "commit"
+        ),
+        Str = "region_fill_frame(" ++
+            FillOpStr ++ "," ++
+            dump_embedded_stack_frame_id(EmbeddedStackFrame) ++ "," ++
+            dump_rval(yes(ProcLabel), IdRval) ++ "," ++
+            dump_lval(yes(ProcLabel), NumLval) ++ "," ++
+            dump_lval(yes(ProcLabel), AddrLval) ++ ")"
+    ;
+        Instr = region_set_fixed_slot(SetOp, EmbeddedStackFrame, ValueRval),
+        (
+            SetOp = region_set_ite_num_protects,
+            SetOpStr = "ite_num_protects"
+        ;
+            SetOp = region_set_ite_num_snapshots,
+            SetOpStr = "ite_num_snapshots"
+        ;
+            SetOp = region_set_disj_num_protects,
+            SetOpStr = "disj_num_protects"
+        ;
+            SetOp = region_set_disj_num_snapshots,
+            SetOpStr = "disj_num_snapshots"
+        ;
+            SetOp = region_set_commit_num_entries,
+            SetOpStr = "commit_num_entries"
+        ),
+        Str = "region_set_fixed_slot(" ++
+            SetOpStr ++ "," ++
+            dump_embedded_stack_frame_id(EmbeddedStackFrame) ++ "," ++
+            dump_rval(yes(ProcLabel), ValueRval) ++ ")"
+    ;
+        Instr = use_and_maybe_pop_region_frame(UseOp, EmbeddedStackFrame),
+        (
+            UseOp = region_ite_then(region_ite_semidet_cond),
+            UseOpStr = "region_ite_then(semidet)"
+        ;
+            UseOp = region_ite_then(region_ite_nondet_cond),
+            UseOpStr = "region_ite_then(nondet)"
+        ;
+            UseOp = region_ite_else(region_ite_semidet_cond),
+            UseOpStr = "region_ite_else(semidet)"
+        ;
+            UseOp = region_ite_else(region_ite_nondet_cond),
+            UseOpStr = "region_ite_else(nondet)"
+        ;
+            UseOp = region_ite_nondet_cond_fail,
+            UseOpStr = "region_ite_nondet_cond_fail"
+        ;
+            UseOp = region_disj_later,
+            UseOpStr = "region_disj_later"
+        ;
+            UseOp = region_disj_last,
+            UseOpStr = "region_disj_last"
+        ;
+            UseOp = region_commit_success,
+            UseOpStr = "region_commit_success"
+        ;
+            UseOp = region_commit_failure,
+            UseOpStr = "region_commit_failure"
+        ),
+        Str = "use_and_maybe_pop_region_frame(" ++
+            UseOpStr ++ "," ++
+            dump_embedded_stack_frame_id(EmbeddedStackFrame) ++ ")"
+    ;
         Instr = store_ticket(Lval),
         Str = "store_ticket(" ++ dump_lval(yes(ProcLabel), Lval) ++ ")"
     ;
@@ -817,6 +909,20 @@
             ++ ")"
     ).
 
+:- func dump_embedded_stack_frame_id(embedded_stack_frame_id) = string.
+
+dump_embedded_stack_frame_id(EmbeddedStackFrame) = Str :-
+    EmbeddedStackFrame = embedded_stack_frame_id(StackId, FirstSlot, LastSlot),
+    (
+        StackId = det_stack,
+        Str = "detstack slots " ++ int_to_string(FirstSlot)
+            ++ ".." ++ int_to_string(LastSlot)
+    ;
+        StackId = nondet_stack,
+        Str = "nondetstack slots " ++ int_to_string(FirstSlot)
+            ++ ".." ++ int_to_string(LastSlot)
+    ).
+
 :- func dump_may_call_mercury(proc_may_call_mercury) = string.
 
 dump_may_call_mercury(proc_may_call_mercury) = "may_call_mercury".
Index: compiler/opt_util.m
===================================================================
RCS file: /home/mercury/mercury1/repository/mercury/compiler/opt_util.m,v
retrieving revision 1.164
diff -u -b -r1.164 opt_util.m
--- compiler/opt_util.m	9 Jul 2007 13:28:33 -0000	1.164
+++ compiler/opt_util.m	30 Jul 2007 05:18:05 -0000
@@ -289,29 +289,31 @@
     % called procedure) are always replaced; references that treat the label
     % as data are replaced iff the third argument is set to "yes".
     %
-    % With replace_labels_instruction_list, the fourth arg says whether
+    % With replace_labels_instruction_list, the last arg says whether
     % it is OK to replace a label in a label instruction itself.
     %
-:- pred replace_labels_instr(instr::in, map(label, label)::in,
-    bool::in, instr::out) is det.
+:- pred replace_labels_instr(instr::in, instr::out,
+    map(label, label)::in, bool::in) is det.
 
-:- pred replace_labels_instruction(instruction::in,
-    map(label, label)::in, bool::in, instruction::out) is det.
+:- pred replace_labels_instruction(instruction::in, instruction::out,
+    map(label, label)::in, bool::in) is det.
 
-:- pred replace_labels_instruction_list(list(instruction)::in,
-    map(label, label)::in, bool::in, bool::in, list(instruction)::out) is det.
+:- pred replace_labels_instruction_list(
+    list(instruction)::in, list(instruction)::out,
+    map(label, label)::in, bool::in, bool::in) is det.
 
-:- pred replace_labels_comps(list(foreign_proc_component)::in,
-    map(label, label)::in, list(foreign_proc_component)::out) is det.
+:- pred replace_labels_comps(
+    list(foreign_proc_component)::in, list(foreign_proc_component)::out,
+    map(label, label)::in) is det.
 
-:- pred replace_labels_code_addr(code_addr::in, map(label, label)::in,
-    code_addr::out) is det.
+:- pred replace_labels_code_addr(code_addr::in, code_addr::out,
+    map(label, label)::in) is det.
 
-:- pred replace_labels_label_list(list(label)::in,
-    map(label, label)::in, list(label)::out) is det.
+:- pred replace_labels_label_list(list(label)::in, list(label)::out,
+    map(label, label)::in) is det.
 
-:- pred replace_labels_label(label::in, map(label, label)::in,
-    label::out) is det.
+:- pred replace_labels_label(label::in, label::out, map(label, label)::in)
+    is det.
 
 %-----------------------------------------------------------------------------%
 %-----------------------------------------------------------------------------%
@@ -860,20 +862,29 @@
         ),
         Refers = no
     ;
+        ( Uinstr = llcall(_, _, _, _, _, _)
+        ; Uinstr = mkframe(_, _)
+        ; Uinstr = push_region_frame(_, _)
+        ; Uinstr = region_fill_frame(_, _, _, _, _)
+        ; Uinstr = region_set_fixed_slot(_, _, _)
+        ; Uinstr = use_and_maybe_pop_region_frame(_, _)
+        ; Uinstr = incr_sp(_, _, _)
+        ; Uinstr = decr_sp(_)
+        ; Uinstr = decr_sp_and_return(_)
+        ; Uinstr = fork(_)
+        ),
+        Refers = yes
+    ;
         Uinstr = block(_, _, BlockInstrs),
         Refers = block_refers_to_stack(BlockInstrs)
     ;
-        Uinstr = assign(Lval, Rval),
+        ( Uinstr = assign(Lval, Rval)
+        ; Uinstr = keep_assign(Lval, Rval)
+        ),
         Refers = bool.or(
             lval_refers_stackvars(Lval),
             rval_refers_stackvars(Rval))
     ;
-        Uinstr = llcall(_, _, _, _, _, _),
-        Refers = yes
-    ;
-        Uinstr = mkframe(_, _),
-        Refers = yes
-    ;
         Uinstr = goto(CodeAddr),
         Refers = code_addr_refers_to_stack(CodeAddr)
     ;
@@ -892,18 +903,15 @@
         Refers = lval_refers_stackvars(Lval)
     ;
         Uinstr = incr_hp(Lval, _, _, Rval, _, _, MaybeRegionRval),
+        Refers0 = bool.or(
+            lval_refers_stackvars(Lval),
+            rval_refers_stackvars(Rval)),
         (
             MaybeRegionRval = no,
-            Refers = bool.or(
-                lval_refers_stackvars(Lval),
-                rval_refers_stackvars(Rval))
+            Refers = Refers0
         ;
             MaybeRegionRval = yes(RegionRval),
-            Refers0 = bool.or(
-                lval_refers_stackvars(Lval),
-                rval_refers_stackvars(Rval)),
-            Refers = bool.or(Refers0,
-                rval_refers_stackvars(RegionRval))
+            Refers = bool.or(Refers0, rval_refers_stackvars(RegionRval))
         )
     ;
         Uinstr = mark_hp(Lval),
@@ -927,15 +935,6 @@
         Uinstr = prune_tickets_to(Rval),
         Refers = rval_refers_stackvars(Rval)
     ;
-        Uinstr = incr_sp(_, _, _),
-        Refers = yes
-    ;
-        Uinstr = decr_sp(_),
-        Refers = yes
-    ;
-        Uinstr = decr_sp_and_return(_),
-        Refers = yes
-    ;
         Uinstr = foreign_proc_code(_, Components, _, _, _, _, _, _, _),
         Refers = bool.or_list(list.map(foreign_proc_component_refers_stackvars,
             Components))
@@ -943,9 +942,6 @@
         Uinstr = init_sync_term(Lval, _),
         Refers = lval_refers_stackvars(Lval)
     ;
-        Uinstr = fork(_),
-        Refers = yes
-    ;
         Uinstr = join_and_continue(Lval, _),
         Refers = lval_refers_stackvars(Lval)
     ).
@@ -1061,6 +1057,7 @@
 can_instr_branch_away(livevals(_)) = no.
 can_instr_branch_away(block(_, _, _)) = yes.
 can_instr_branch_away(assign(_, _)) = no.
+can_instr_branch_away(keep_assign(_, _)) = no.
 can_instr_branch_away(llcall(_, _, _, _, _, _)) = yes.
 can_instr_branch_away(mkframe(_, _)) = no.
 can_instr_branch_away(label(_)) = no.
@@ -1074,6 +1071,10 @@
 can_instr_branch_away(mark_hp(_)) = no.
 can_instr_branch_away(restore_hp(_)) = no.
 can_instr_branch_away(free_heap(_)) = no.
+can_instr_branch_away(push_region_frame(_, _)) = no.
+can_instr_branch_away(region_fill_frame(_, _, _, _, _)) = no.
+can_instr_branch_away(region_set_fixed_slot(_, _, _)) = no.
+can_instr_branch_away(use_and_maybe_pop_region_frame(_, _)) = no.
 can_instr_branch_away(store_ticket(_)) = no.
 can_instr_branch_away(reset_ticket(_, _)) = no.
 can_instr_branch_away(discard_ticket) = no.
@@ -1135,6 +1136,7 @@
 can_instr_fall_through(block(_, _, Instrs)) = FallThrough :-
     can_block_fall_through(Instrs, FallThrough).
 can_instr_fall_through(assign(_, _)) = yes.
+can_instr_fall_through(keep_assign(_, _)) = yes.
 can_instr_fall_through(llcall(_, _, _, _, _, _)) = no.
 can_instr_fall_through(mkframe(_, _)) = yes.
 can_instr_fall_through(label(_)) = yes.
@@ -1148,6 +1150,10 @@
 can_instr_fall_through(mark_hp(_)) = yes.
 can_instr_fall_through(restore_hp(_)) = yes.
 can_instr_fall_through(free_heap(_)) = yes.
+can_instr_fall_through(push_region_frame(_, _)) = yes.
+can_instr_fall_through(region_fill_frame(_, _, _, _, _)) = yes.
+can_instr_fall_through(region_set_fixed_slot(_, _, _)) = yes.
+can_instr_fall_through(use_and_maybe_pop_region_frame(_, _)) = yes.
 can_instr_fall_through(store_ticket(_)) = yes.
 can_instr_fall_through(reset_ticket(_, _)) = yes.
 can_instr_fall_through(discard_ticket) = yes.
@@ -1181,6 +1187,7 @@
 can_use_livevals(livevals(_), no).
 can_use_livevals(block(_, _, _), no).
 can_use_livevals(assign(_, _), no).
+can_use_livevals(keep_assign(_, _), no).
 can_use_livevals(llcall(_, _, _, _, _, _), yes).
 can_use_livevals(mkframe(_, _), no).
 can_use_livevals(label(_), no).
@@ -1194,6 +1201,10 @@
 can_use_livevals(mark_hp(_), no).
 can_use_livevals(restore_hp(_), no).
 can_use_livevals(free_heap(_), no).
+can_use_livevals(push_region_frame(_, _), no).
+can_use_livevals(region_fill_frame(_, _, _, _, _), no).
+can_use_livevals(region_set_fixed_slot(_, _, _), no).
+can_use_livevals(use_and_maybe_pop_region_frame(_, _), no).
 can_use_livevals(store_ticket(_), no).
 can_use_livevals(reset_ticket(_, _), no).
 can_use_livevals(discard_ticket, no).
@@ -1243,6 +1254,7 @@
 instr_labels_2(block(_, _, Instrs), Labels, CodeAddrs) :-
     instr_list_labels(Instrs, Labels, CodeAddrs).
 instr_labels_2(assign(_,_), [], []).
+instr_labels_2(keep_assign(_,_), [], []).
 instr_labels_2(llcall(Target, Ret, _, _, _, _), [], [Target, Ret]).
 instr_labels_2(mkframe(_, yes(Addr)), [], [Addr]).
 instr_labels_2(mkframe(_, no), [], []).
@@ -1257,6 +1269,10 @@
 instr_labels_2(mark_hp(_), [], []).
 instr_labels_2(restore_hp(_), [], []).
 instr_labels_2(free_heap(_), [], []).
+instr_labels_2(push_region_frame(_, _), [], []).
+instr_labels_2(region_fill_frame(_, _, _, _, _), [], []).
+instr_labels_2(region_set_fixed_slot(_, _, _), [], []).
+instr_labels_2(use_and_maybe_pop_region_frame(_, _), [], []).
 instr_labels_2(store_ticket(_), [], []).
 instr_labels_2(reset_ticket(_, _), [], []).
 instr_labels_2(discard_ticket, [], []).
@@ -1284,6 +1300,7 @@
 possible_targets(block(_, _, _), _, _) :-
     unexpected(this_file, "block in possible_targets").
 possible_targets(assign(_, _), [], []).
+possible_targets(keep_assign(_, _), [], []).
 possible_targets(llcall(_, Return, _, _, _, _), Labels, CodeAddrs) :-
     ( Return = code_label(ReturnLabel) ->
         Labels = [ReturnLabel],
@@ -1318,6 +1335,10 @@
 possible_targets(mark_hp(_), [], []).
 possible_targets(restore_hp(_), [], []).
 possible_targets(free_heap(_), [], []).
+possible_targets(push_region_frame(_, _), [], []).
+possible_targets(region_fill_frame(_, _, _, _, _), [], []).
+possible_targets(region_set_fixed_slot(_, _, _), [], []).
+possible_targets(use_and_maybe_pop_region_frame(_, _), [], []).
 possible_targets(store_ticket(_), [], []).
 possible_targets(reset_ticket(_, _), [], []).
 possible_targets(discard_ticket, [], []).
@@ -1378,6 +1399,7 @@
 instr_rvals_and_lvals(block(_, _, Instrs), Labels, CodeAddrs) :-
     instr_list_rvals_and_lvals(Instrs, Labels, CodeAddrs).
 instr_rvals_and_lvals(assign(Lval,Rval), [Rval], [Lval]).
+instr_rvals_and_lvals(keep_assign(Lval,Rval), [Rval], [Lval]).
 instr_rvals_and_lvals(llcall(_, _, _, _, _, _), [], []).
 instr_rvals_and_lvals(mkframe(_, _), [], []).
 instr_rvals_and_lvals(label(_), [], []).
@@ -1399,6 +1421,16 @@
 instr_rvals_and_lvals(mark_hp(Lval), [], [Lval]).
 instr_rvals_and_lvals(restore_hp(Rval), [Rval], []).
 instr_rvals_and_lvals(free_heap(Rval), [Rval], []).
+    % The region instructions implicitly specify some stackvars or framevars,
+    % but they cannot reference lvals or rvals that involve code addresses or
+    % labels, and that is the motivation of the only current invoker of
+    % instr_rvals_and_lvals.
+instr_rvals_and_lvals(push_region_frame(_, _), [], []).
+instr_rvals_and_lvals(region_fill_frame(_, _, IdRval, NumLval, AddrLval),
+    [IdRval], [NumLval, AddrLval]).
+instr_rvals_and_lvals(region_set_fixed_slot(_, _, ValueRval),
+    [ValueRval], []).
+instr_rvals_and_lvals(use_and_maybe_pop_region_frame(_, _), [], []).
 instr_rvals_and_lvals(store_ticket(Lval), [], [Lval]).
 instr_rvals_and_lvals(reset_ticket(Rval, _Reason), [Rval], []).
 instr_rvals_and_lvals(discard_ticket, [], []).
@@ -1518,6 +1550,9 @@
 count_temps_instr(assign(Lval, Rval), !R, !F) :-
     count_temps_lval(Lval, !R, !F),
     count_temps_rval(Rval, !R, !F).
+count_temps_instr(keep_assign(Lval, Rval), !R, !F) :-
+    count_temps_lval(Lval, !R, !F),
+    count_temps_rval(Rval, !R, !F).
 count_temps_instr(llcall(_, _, _, _, _, _), !R, !F).
 count_temps_instr(mkframe(_, _), !R, !F).
 count_temps_instr(label(_), !R, !F).
@@ -1546,6 +1581,17 @@
     count_temps_rval(Rval, !R, !F).
 count_temps_instr(free_heap(Rval), !R, !F) :-
     count_temps_rval(Rval, !R, !F).
+count_temps_instr(push_region_frame(_StackId, _EmbeddedStackFrame), !R, !F).
+count_temps_instr(region_fill_frame(_FillOp, _EmbeddedStackFrame, IdRval,
+        NumLval, AddrLval), !R, !F) :-
+    count_temps_rval(IdRval, !R, !F),
+    count_temps_lval(NumLval, !R, !F),
+    count_temps_lval(AddrLval, !R, !F).
+count_temps_instr(region_set_fixed_slot(_SetlOp, _EmbeddedStackFrame,
+        ValueRval), !R, !F) :-
+    count_temps_rval(ValueRval, !R, !F).
+count_temps_instr(use_and_maybe_pop_region_frame(_UseOp, _EmbeddedStackFrame),
+        !R, !F).
 count_temps_instr(store_ticket(Lval), !R, !F) :-
     count_temps_lval(Lval, !R, !F).
 count_temps_instr(reset_ticket(Rval, _Reason), !R, !F) :-
@@ -1619,7 +1665,21 @@
     is det.
 
 count_temps_lval(Lval, !R, !F) :-
-    ( Lval = temp(Type, N) ->
+    (
+        ( Lval = reg(_, _)
+        ; Lval = succip
+        ; Lval = maxfr
+        ; Lval = curfr
+        ; Lval = hp
+        ; Lval = sp
+        ; Lval = parent_sp
+        ; Lval = stackvar(_)
+        ; Lval = framevar(_)
+        ; Lval = parent_stackvar(_)
+        ; Lval = global_var_ref(_)
+        )
+    ;
+        Lval = temp(Type, N),
         (
             Type = reg_r,
             int.max(N, !R)
@@ -1627,19 +1687,65 @@
             Type = reg_f,
             int.max(N, !F)
         )
-    ; Lval = field(_, Rval, FieldNum) ->
-        count_temps_rval(Rval, !R, !F),
-        count_temps_rval(FieldNum, !R, !F)
     ;
-        true
+        Lval = field(_, BaseAddrRval, FieldNumRval),
+        count_temps_rval(BaseAddrRval, !R, !F),
+        count_temps_rval(FieldNumRval, !R, !F)
+    ;
+        ( Lval = succip_slot(Rval)
+        ; Lval = succfr_slot(Rval)
+        ; Lval = redoip_slot(Rval)
+        ; Lval = redofr_slot(Rval)
+        ; Lval = prevfr_slot(Rval)
+        ; Lval = mem_ref(Rval)
+        ),
+        count_temps_rval(Rval, !R, !F)
+    ;
+        Lval = lvar(_),
+        unexpected(this_file, "count_temps_lval: lvar")
     ).
 
 :- pred count_temps_rval(rval::in, int::in, int::out, int::in, int::out)
     is det.
 
-% XXX We assume that we don't generate code
-% that uses a temp var without defining it.
-count_temps_rval(_, !R, !F).
+count_temps_rval(Rval, !R, !F) :-
+    (
+        Rval = lval(Lval),
+        count_temps_lval(Lval, !R, !F)
+    ;
+        Rval = var(_),
+        unexpected(this_file, "count_temps_rval: var")
+    ;
+        Rval = mkword(_Tag, SubRval),
+        count_temps_rval(SubRval, !R, !F)
+    ;
+        Rval = const(_Const)
+    ;
+        Rval = unop(_Unop, SubRvalA),
+        count_temps_rval(SubRvalA, !R, !F)
+    ;
+        Rval = binop(_Binop, SubRvalA, SubRvalB),
+        count_temps_rval(SubRvalA, !R, !F),
+        count_temps_rval(SubRvalB, !R, !F)
+    ;
+        Rval = mem_addr(MemRef),
+        count_temps_mem_ref(MemRef, !R, !F)
+    ).
+
+:- pred count_temps_mem_ref(mem_ref::in, int::in, int::out, int::in, int::out)
+    is det.
+
+count_temps_mem_ref(MemRef, !R, !F) :-
+    (
+        ( MemRef = stackvar_ref(Rval)
+        ; MemRef = framevar_ref(Rval)
+        ),
+        count_temps_rval(Rval, !R, !F)
+    ;
+        MemRef = heap_ref(CellRval, _Tag, FieldNumRval),
+        count_temps_rval(CellRval, !R, !F),
+        count_temps_rval(FieldNumRval, !R, !F)
+    ).
 
 format_label(internal_label(_, ProcLabel)) = format_proc_label(ProcLabel).
 format_label(entry_label(_, ProcLabel)) = format_proc_label(ProcLabel).
@@ -1692,13 +1798,8 @@
         ( Uinstr = comment(_)
         ; Uinstr = livevals(_)
         ; Uinstr = label(_)
-        ; Uinstr = free_heap(_)
-        ; Uinstr = store_ticket(_)
-        ; Uinstr = reset_ticket(_, _)
         ; Uinstr = prune_ticket
         ; Uinstr = discard_ticket
-        ; Uinstr = prune_tickets_to(_)
-        ; Uinstr = mark_ticket_stack(_)
         ; Uinstr = incr_sp(_, _, _)
         ; Uinstr = decr_sp(_)
         ; Uinstr = decr_sp_and_return(_)
@@ -1723,23 +1824,24 @@
         % Blocks aren't introduced until after the last user of this predicate.
         unexpected(this_file, "touches_nondet_ctrl_instr: block")
     ;
-        Uinstr = assign(Lval, Rval),
+        ( Uinstr = assign(Lval, Rval)
+        ; Uinstr = keep_assign(Lval, Rval)
+        ),
         TouchLval = touches_nondet_ctrl_lval(Lval),
         TouchRval = touches_nondet_ctrl_rval(Rval),
         bool.or(TouchLval, TouchRval, Touch)
     ;
         Uinstr = incr_hp(Lval, _, _, Rval, _, _, MaybeRegionRval),
-        TouchLval = touches_nondet_ctrl_lval(Lval),
-        TouchRval = touches_nondet_ctrl_rval(Rval),
+        Touch0 = bool.or(
+            touches_nondet_ctrl_lval(Lval),
+            touches_nondet_ctrl_rval(Rval)),
         (
             MaybeRegionRval = yes(RegionRval),
-            TouchRegionRval = touches_nondet_ctrl_rval(RegionRval)
+            Touch = bool.or(Touch0, touches_nondet_ctrl_rval(RegionRval))
         ;
             MaybeRegionRval = no,
-            TouchRegionRval = no
-        ),
-        bool.or(TouchLval, TouchRval, Touch0),
-        bool.or(Touch0, TouchRegionRval, Touch)
+            Touch = Touch0
+        )
     ;
         Uinstr = mark_hp(Lval),
         Touch = touches_nondet_ctrl_lval(Lval)
@@ -1747,6 +1849,38 @@
         Uinstr = restore_hp(Rval),
         Touch = touches_nondet_ctrl_rval(Rval)
     ;
+        Uinstr = free_heap(Rval),
+        Touch = touches_nondet_ctrl_rval(Rval)
+    ;
+        Uinstr = push_region_frame(_StackId, _EmbeddedStackFrame),
+        Touch = no
+    ;
+        Uinstr = region_fill_frame(_FillOp, _EmbeddedStackFrame, IdRval,
+            NumLval, AddrLval),
+        Touch = bool.or(
+            touches_nondet_ctrl_rval(IdRval),
+            bool.or(
+                touches_nondet_ctrl_lval(NumLval),
+                touches_nondet_ctrl_lval(AddrLval)))
+    ;
+        Uinstr = region_set_fixed_slot(_SetOp, _EmbeddedStackFrame, ValueRval),
+        Touch = touches_nondet_ctrl_rval(ValueRval)
+    ;
+        Uinstr = use_and_maybe_pop_region_frame(_UseOp, _EmbeddedStackFrame),
+        Touch = no
+    ;
+        Uinstr = store_ticket(Lval),
+        Touch = touches_nondet_ctrl_lval(Lval)
+    ;
+        Uinstr = reset_ticket(Rval, _),
+        Touch = touches_nondet_ctrl_rval(Rval)
+    ;
+        Uinstr = mark_ticket_stack(Lval),
+        Touch = touches_nondet_ctrl_lval(Lval)
+    ;
+        Uinstr = prune_tickets_to(Rval),
+        Touch = touches_nondet_ctrl_rval(Rval)
+    ;
         Uinstr = foreign_proc_code(_, Components, _, _, _, _, _, _, _),
         Touch = touches_nondet_ctrl_components(Components)
     ).
@@ -1922,52 +2056,78 @@
 % The code in this section is concerned with replacing all references
 % to one given label with a reference to another given label.
 
-replace_labels_instruction_list([], _, _, _, []).
-replace_labels_instruction_list([Instr0 | Instrs0], ReplMap, ReplData,
-        ReplLabel, [Instr | Instrs]) :-
+replace_labels_instruction_list([], [], _, _, _).
+replace_labels_instruction_list([Instr0 | Instrs0], [Instr | Instrs],
+        ReplMap, ReplData, ReplLabel) :-
     (
         Instr0 = llds_instr(label(InstrLabel), Comment),
         ReplLabel = yes
     ->
-        replace_labels_label(InstrLabel, ReplMap, ReplInstrLabel),
+        replace_labels_label(InstrLabel, ReplInstrLabel, ReplMap),
         Instr = llds_instr(label(ReplInstrLabel), Comment)
     ;
-        replace_labels_instruction(Instr0, ReplMap, ReplData, Instr)
+        replace_labels_instruction(Instr0, Instr, ReplMap, ReplData)
     ),
-    replace_labels_instruction_list(Instrs0, ReplMap, ReplData, ReplLabel,
-        Instrs).
+    replace_labels_instruction_list(Instrs0, Instrs,
+        ReplMap, ReplData, ReplLabel).
+
+replace_labels_instruction(Instr0, Instr, ReplMap, ReplData) :-
+    Instr0 = llds_instr(Uinstr0, Comment),
+    replace_labels_instr(Uinstr0, Uinstr, ReplMap, ReplData),
+    Instr = llds_instr(Uinstr, Comment).
 
-replace_labels_instruction(llds_instr(Instr0, Comment), ReplMap, ReplData,
-        llds_instr(Instr, Comment)) :-
-    replace_labels_instr(Instr0, ReplMap, ReplData, Instr).
-
-replace_labels_instr(comment(Comment), _, _, comment(Comment)).
-replace_labels_instr(livevals(Livevals), _, _, livevals(Livevals)).
-replace_labels_instr(block(R, F, Instrs0), ReplMap, ReplData,
-        block(R, F, Instrs)) :-
+replace_labels_instr(Uinstr0, Uinstr, ReplMap, ReplData) :-
+    (
+        ( Uinstr0 = comment(_)
+        ; Uinstr0 = livevals(_)
+        ; Uinstr0 = discard_ticket
+        ; Uinstr0 = prune_ticket
+        ; Uinstr0 = incr_sp(_, _, _)
+        ; Uinstr0 = decr_sp(_)
+        ; Uinstr0 = decr_sp_and_return(_)
+        ),
+        Uinstr = Uinstr0
+    ;
+        Uinstr0 = block(R, F, Instrs0),
     % There should be no labels in Instrs0.
-    replace_labels_instruction_list(Instrs0, ReplMap, ReplData, no, Instrs).
-replace_labels_instr(assign(Lval0, Rval0), ReplMap, ReplData,
-        assign(Lval, Rval)) :-
+        replace_labels_instruction_list(Instrs0, Instrs,
+            ReplMap, ReplData, no),
+        Uinstr = block(R, F, Instrs)
+    ;
+        Uinstr0 = assign(Lval0, Rval0),
     (
         ReplData = yes,
-        replace_labels_lval(Lval0, ReplMap, Lval),
-        replace_labels_rval(Rval0, ReplMap, Rval)
+            replace_labels_lval(Lval0, Lval, ReplMap),
+            replace_labels_rval(Rval0, Rval, ReplMap)
     ;
         ReplData = no,
         Lval = Lval0,
         Rval = Rval0
-    ).
-replace_labels_instr(llcall(Target, Return0, LiveInfo, CXT, GP, CM),
-        ReplMap, _, llcall(Target, Return, LiveInfo, CXT, GP, CM)) :-
-    replace_labels_code_addr(Return0, ReplMap, Return).
-replace_labels_instr(mkframe(NondetFrameInfo, MaybeRedoip0), ReplMap,
-        ReplData, mkframe(NondetFrameInfo, MaybeRedoip)) :-
+        ),
+        Uinstr = assign(Lval, Rval)
+    ;
+        Uinstr0 = keep_assign(Lval0, Rval0),
+        (
+            ReplData = yes,
+            replace_labels_lval(Lval0, Lval, ReplMap),
+            replace_labels_rval(Rval0, Rval, ReplMap)
+        ;
+            ReplData = no,
+            Lval = Lval0,
+            Rval = Rval0
+        ),
+        Uinstr = keep_assign(Lval, Rval)
+    ;
+        Uinstr0 = llcall(Target, Return0, LiveInfo, CXT, GP, CM),
+        replace_labels_code_addr(Return0, Return, ReplMap),
+        Uinstr = llcall(Target, Return, LiveInfo, CXT, GP, CM)
+    ;
+        Uinstr0 = mkframe(NondetFrameInfo, MaybeRedoip0),
     (
         ReplData = yes,
         (
             MaybeRedoip0 = yes(Redoip0),
-            replace_labels_code_addr(Redoip0, ReplMap, Redoip),
+                replace_labels_code_addr(Redoip0, Redoip, ReplMap),
             MaybeRedoip = yes(Redoip)
         ;
             MaybeRedoip0 = no,
@@ -1976,8 +2136,10 @@
     ;
         ReplData = no,
         MaybeRedoip = MaybeRedoip0
-    ).
-replace_labels_instr(label(Label), ReplMap, _, label(Label)) :-
+        ),
+        Uinstr = mkframe(NondetFrameInfo, MaybeRedoip)
+    ;
+        Uinstr0 = label(Label),
     ( map.search(ReplMap, Label, _) ->
         % The reason why we are replacing references to this label is that
         % it is being eliminated, and in fact should have been already
@@ -1985,59 +2147,74 @@
         unexpected(this_file, "eliminated label in replace_labels_instr")
     ;
         true
-    ).
-replace_labels_instr(goto(Target0), ReplMap, _, goto(Target)) :-
-    replace_labels_code_addr(Target0, ReplMap, Target).
-replace_labels_instr(computed_goto(Rval0, Labels0), ReplMap,
-        ReplData, computed_goto(Rval, Labels)) :-
+        ),
+        Uinstr = label(Label)
+    ;
+        Uinstr0 = goto(Target0),
+        replace_labels_code_addr(Target0, Target, ReplMap),
+        Uinstr = goto(Target)
+    ;
+        Uinstr0 = computed_goto(Rval0, Labels0),
     (
         ReplData = yes,
-        replace_labels_rval(Rval0, ReplMap, Rval)
+            replace_labels_rval(Rval0, Rval, ReplMap)
     ;
         ReplData = no,
         Rval = Rval0
     ),
-    replace_labels_label_list(Labels0, ReplMap, Labels).
-replace_labels_instr(arbitrary_c_code(Lvals, AffectsLiveness, Code), _, _,
-        arbitrary_c_code(Lvals, AffectsLiveness, Code)).
-replace_labels_instr(if_val(Rval0, Target0), ReplMap, ReplData,
-        if_val(Rval, Target)) :-
+        replace_labels_label_list(Labels0, Labels, ReplMap),
+        Uinstr = computed_goto(Rval, Labels)
+    ;
+        Uinstr0 = arbitrary_c_code(AffectsLiveness, Lvals0, Code),
     (
         ReplData = yes,
-        replace_labels_rval(Rval0, ReplMap, Rval)
+            replace_labels_c_code_live_lvals(Lvals0, Lvals, ReplMap)
+        ;
+            ReplData = no,
+            Lvals = Lvals0
+        ),
+        Uinstr = arbitrary_c_code(AffectsLiveness, Lvals, Code)
+    ;
+        Uinstr0 = if_val(Rval0, Target0),
+        (
+            ReplData = yes,
+            replace_labels_rval(Rval0, Rval, ReplMap)
     ;
         ReplData = no,
         Rval = Rval0
     ),
-    replace_labels_code_addr(Target0, ReplMap, Target).
-replace_labels_instr(save_maxfr(Lval0), ReplMap, ReplData, save_maxfr(Lval)) :-
+        replace_labels_code_addr(Target0, Target, ReplMap),
+        Uinstr = if_val(Rval, Target)
+    ;
+        Uinstr0 = save_maxfr(Lval0),
     (
         ReplData = yes,
-        replace_labels_lval(Lval0, ReplMap, Lval)
+            replace_labels_lval(Lval0, Lval, ReplMap)
     ;
         ReplData = no,
         Lval = Lval0
-    ).
-replace_labels_instr(restore_maxfr(Lval0), ReplMap, ReplData,
-        restore_maxfr(Lval)) :-
+        ),
+        Uinstr = save_maxfr(Lval)
+    ;
+        Uinstr0 = restore_maxfr(Lval0),
     (
         ReplData = yes,
-        replace_labels_lval(Lval0, ReplMap, Lval)
+            replace_labels_lval(Lval0, Lval, ReplMap)
     ;
         ReplData = no,
         Lval = Lval0
-    ).
-replace_labels_instr(
-        incr_hp(Lval0, MaybeTag, MO, Rval0, Msg, Atomic, MaybeRegionRval0),
-        ReplMap, ReplData,
-        incr_hp(Lval, MaybeTag, MO, Rval, Msg, Atomic, MaybeRegionRval)) :-
+        ),
+        Uinstr = restore_maxfr(Lval)
+    ;
+        Uinstr0 = incr_hp(Lval0, MaybeTag, MO, Rval0, Msg, Atomic,
+            MaybeRegionRval0),
     (
         ReplData = yes,
-        replace_labels_lval(Lval0, ReplMap, Lval),
-        replace_labels_rval(Rval0, ReplMap, Rval),
+            replace_labels_lval(Lval0, Lval, ReplMap),
+            replace_labels_rval(Rval0, Rval, ReplMap),
         (
             MaybeRegionRval0 = yes(RegionRval0),
-            replace_labels_rval(RegionRval0, ReplMap, RegionRval),
+                replace_labels_rval(RegionRval0, RegionRval, ReplMap),
             MaybeRegionRval = yes(RegionRval)
         ;
             MaybeRegionRval0 = no,
@@ -2048,91 +2225,140 @@
         Lval = Lval0,
         Rval = Rval0,
         MaybeRegionRval = MaybeRegionRval0
-    ).
-replace_labels_instr(mark_hp(Lval0), ReplMap, ReplData,
-        mark_hp(Lval)) :-
+        ),
+        Uinstr = incr_hp(Lval, MaybeTag, MO, Rval, Msg, Atomic,
+            MaybeRegionRval)
+    ;
+        Uinstr0 = mark_hp(Lval0),
     (
         ReplData = yes,
-        replace_labels_lval(Lval0, ReplMap, Lval)
+            replace_labels_lval(Lval0, Lval, ReplMap)
     ;
         ReplData = no,
         Lval = Lval0
-    ).
-replace_labels_instr(restore_hp(Rval0), ReplMap, ReplData,
-        restore_hp(Rval)) :-
+        ),
+        Uinstr = mark_hp(Lval)
+    ;
+        Uinstr0 = restore_hp(Rval0),
     (
         ReplData = yes,
-        replace_labels_rval(Rval0, ReplMap, Rval)
+            replace_labels_rval(Rval0, Rval, ReplMap)
     ;
         ReplData = no,
         Rval = Rval0
-    ).
-replace_labels_instr(free_heap(Rval0), ReplMap, ReplData,
-        free_heap(Rval)) :-
+        ),
+        Uinstr = restore_hp(Rval)
+    ;
+        Uinstr0 = free_heap(Rval0),
     (
         ReplData = yes,
-        replace_labels_rval(Rval0, ReplMap, Rval)
+            replace_labels_rval(Rval0, Rval, ReplMap)
     ;
         ReplData = no,
         Rval = Rval0
-    ).
-replace_labels_instr(store_ticket(Lval0), ReplMap, ReplData,
-        store_ticket(Lval)) :-
+        ),
+        Uinstr = free_heap(Rval)
+    ;
+        Uinstr0 = push_region_frame(StackId, EmbeddedStackFrame),
+        Uinstr = push_region_frame(StackId, EmbeddedStackFrame)
+    ;
+        Uinstr0 = region_fill_frame(FillOp, EmbeddedStackFrame, IdRval0,
+            NumLval0, AddrLval0),
+        (
+            ReplData = yes,
+            replace_labels_rval(IdRval0, IdRval, ReplMap),
+            replace_labels_lval(NumLval0, NumLval, ReplMap),
+            replace_labels_lval(AddrLval0, AddrLval, ReplMap)
+        ;
+            ReplData = no,
+            IdRval = IdRval0,
+            NumLval = NumLval0,
+            AddrLval = AddrLval0
+        ),
+        Uinstr = region_fill_frame(FillOp, EmbeddedStackFrame, IdRval,
+            NumLval, AddrLval)
+    ;
+        Uinstr0 = region_set_fixed_slot(SetOp, EmbeddedStackFrame,
+            ValueRval0),
+        (
+            ReplData = yes,
+            replace_labels_rval(ValueRval0, ValueRval, ReplMap)
+        ;
+            ReplData = no,
+            ValueRval = ValueRval0
+        ),
+        Uinstr = region_set_fixed_slot(SetOp, EmbeddedStackFrame,
+            ValueRval)
+    ;
+        Uinstr0 = use_and_maybe_pop_region_frame(UseOp, EmbeddedStackFrame),
+        Uinstr = use_and_maybe_pop_region_frame(UseOp, EmbeddedStackFrame)
+    ;
+        Uinstr0 = store_ticket(Lval0),
     (
         ReplData = yes,
-        replace_labels_lval(Lval0, ReplMap, Lval)
+            replace_labels_lval(Lval0, Lval, ReplMap)
     ;
         ReplData = no,
         Lval = Lval0
-    ).
-replace_labels_instr(reset_ticket(Rval0, Reason), ReplMap, ReplData,
-        reset_ticket(Rval, Reason)) :-
+        ),
+        Uinstr = store_ticket(Lval)
+    ;
+        Uinstr0 = reset_ticket(Rval0, Reason),
     (
         ReplData = yes,
-        replace_labels_rval(Rval0, ReplMap, Rval)
+            replace_labels_rval(Rval0, Rval, ReplMap)
     ;
         ReplData = no,
         Rval = Rval0
-    ).
-replace_labels_instr(discard_ticket, _, _, discard_ticket).
-replace_labels_instr(prune_ticket, _, _, prune_ticket).
-replace_labels_instr(mark_ticket_stack(Lval0), ReplMap, ReplData,
-        mark_ticket_stack(Lval)) :-
+        ),
+        Uinstr = reset_ticket(Rval, Reason)
+    ;
+        Uinstr0 = mark_ticket_stack(Lval0),
     (
         ReplData = yes,
-        replace_labels_lval(Lval0, ReplMap, Lval)
+            replace_labels_lval(Lval0, Lval, ReplMap)
     ;
         ReplData = no,
         Lval = Lval0
-    ).
-replace_labels_instr(prune_tickets_to(Rval0), ReplMap, ReplData,
-        prune_tickets_to(Rval)) :-
+        ),
+        Uinstr = mark_ticket_stack(Lval)
+    ;
+        Uinstr0 = prune_tickets_to(Rval0),
     (
         ReplData = yes,
-        replace_labels_rval(Rval0, ReplMap, Rval)
+            replace_labels_rval(Rval0, Rval, ReplMap)
     ;
         ReplData = no,
         Rval = Rval0
-    ).
-replace_labels_instr(incr_sp(Size, Msg, Kind), _, _, incr_sp(Size, Msg, Kind)).
-replace_labels_instr(decr_sp(Size), _, _, decr_sp(Size)).
-replace_labels_instr(decr_sp_and_return(Size), _, _, decr_sp_and_return(Size)).
-replace_labels_instr(init_sync_term(T, N), _, _, init_sync_term(T, N)).
-replace_labels_instr(fork(Child0), Replmap, _, fork(Child)) :-
-    replace_labels_label(Child0, Replmap, Child).
-replace_labels_instr(join_and_continue(Lval0, Label0),
-        Replmap, _, join_and_continue(Lval, Label)) :-
-    replace_labels_label(Label0, Replmap, Label),
-    replace_labels_lval(Lval0, Replmap, Lval).
-replace_labels_instr(foreign_proc_code(A, Comps0, C, MaybeFix, MaybeLayout,
-        MaybeOnlyLayout, MaybeSub0, H, I), ReplMap, _,
-        foreign_proc_code(A, Comps, C, MaybeFix, MaybeLayout, MaybeOnlyLayout,
-            MaybeSub, H, I)) :-
+        ),
+        Uinstr = prune_tickets_to(Rval)
+    ;
+        Uinstr0 = init_sync_term(Lval0, N),
+        (
+            ReplData = yes,
+            replace_labels_lval(Lval0, Lval, ReplMap)
+        ;
+            ReplData = no,
+            Lval = Lval0
+        ),
+        Uinstr = init_sync_term(Lval, N)
+    ;
+        Uinstr0 = fork(Child0),
+        replace_labels_label(Child0, Child, ReplMap),
+        Uinstr = fork(Child)
+    ;
+        Uinstr0 = join_and_continue(Lval0, Label0),
+        replace_labels_label(Label0, Label, ReplMap),
+        replace_labels_lval(Lval0, Lval, ReplMap),
+        Uinstr = join_and_continue(Lval, Label)
+    ;
+        Uinstr0 = foreign_proc_code(A, Comps0, C, MaybeFix, MaybeLayout,
+            MaybeOnlyLayout, MaybeSub0, H, I),
     (
         MaybeFix = no
     ;
         MaybeFix = yes(FixLabel0),
-        replace_labels_label(FixLabel0, ReplMap, FixLabel),
+            replace_labels_label(FixLabel0, FixLabel, ReplMap),
             % We cannot replace the label in the C code string itself.
         expect(unify(FixLabel0, FixLabel), this_file,
             "trying to replace Mercury label in C code")
@@ -2141,7 +2367,7 @@
         MaybeLayout = no
     ;
         MaybeLayout = yes(LayoutLabel0),
-        replace_labels_label(LayoutLabel0, ReplMap, LayoutLabel),
+            replace_labels_label(LayoutLabel0, LayoutLabel, ReplMap),
             % We cannot replace a label that has a layout structure.
         expect(unify(LayoutLabel0, LayoutLabel), this_file,
             "trying to replace Mercury label with layout")
@@ -2150,7 +2376,7 @@
         MaybeOnlyLayout = no
     ;
         MaybeOnlyLayout = yes(OnlyLayoutLabel0),
-        replace_labels_label(OnlyLayoutLabel0, ReplMap, OnlyLayoutLabel),
+            replace_labels_label(OnlyLayoutLabel0, OnlyLayoutLabel, ReplMap),
             % We cannot replace a label that has a layout structure.
         expect(unify(OnlyLayoutLabel0, OnlyLayoutLabel), this_file,
             "trying to replace Mercury label with layout")
@@ -2161,130 +2387,207 @@
         Comps = Comps0
     ;
         MaybeSub0 = yes(SubLabel0),
-        replace_labels_label(SubLabel0, ReplMap, SubLabel),
+            replace_labels_label(SubLabel0, SubLabel, ReplMap),
         MaybeSub = yes(SubLabel),
-        replace_labels_comps(Comps0, ReplMap, Comps)
+            replace_labels_comps(Comps0, Comps, ReplMap)
+        ),
+        Uinstr = foreign_proc_code(A, Comps, C, MaybeFix, MaybeLayout,
+            MaybeOnlyLayout, MaybeSub, H, I)
+    ).
+
+replace_labels_comps([], [], _).
+replace_labels_comps([Comp0 | Comps0], [Comp | Comps], ReplMap) :-
+    replace_labels_comp(Comp0, Comp, ReplMap),
+    replace_labels_comps(Comps0, Comps, ReplMap).
+
+:- pred replace_labels_comp(
+    foreign_proc_component::in, foreign_proc_component::out,
+    map(label, label)::in) is det.
+
+replace_labels_comp(Comp0, Comp, ReplMap) :-
+    (
+        ( Comp0 = foreign_proc_inputs(_)
+        ; Comp0 = foreign_proc_outputs(_)
+        ; Comp0 = foreign_proc_user_code(_, _, _)
+        ; Comp0 = foreign_proc_raw_code(_, _, _, _)
+        ; Comp0 = foreign_proc_noop
+        ),
+        Comp = Comp0
+    ;
+        Comp0 = foreign_proc_fail_to(Label0),
+        replace_labels_label(Label0, Label, ReplMap),
+        Comp = foreign_proc_fail_to(Label)
+    ).
+
+:- pred replace_labels_c_code_live_lvals(
+    c_code_live_lvals::in, c_code_live_lvals::out, map(label, label)::in)
+    is det.
+
+replace_labels_c_code_live_lvals(LiveLvals0, LiveLvals, ReplMap) :-
+    (
+        LiveLvals0 = no_live_lvals_info,
+        LiveLvals = LiveLvals0
+    ;
+        LiveLvals0 = live_lvals_info(LvalSet0),
+        set.to_sorted_list(LvalSet0, Lvals0),
+        list.map(replace_labels_lval_map(ReplMap), Lvals0, Lvals),
+        % We cannot replace the lvals inside the C code.
+        expect(unify(Lvals0, Lvals), this_file,
+            "replace_labels_c_code_live_lvals: some replacements"),
+        LiveLvals = LiveLvals0
+    ).
+
+:- pred replace_labels_lval_map(map(label, label)::in, lval::in, lval::out)
+    is det.
+
+replace_labels_lval_map(ReplMap, Lval0, Lval) :-
+    replace_labels_lval(Lval0, Lval, ReplMap).
+
+:- pred replace_labels_lval(lval::in, lval::out, map(label, label)::in) is det.
+
+replace_labels_lval(Lval0, Lval, ReplMap) :-
+    (
+        ( Lval0 = reg(_, _)
+        ; Lval0 = temp(_, _)
+        ; Lval0 = stackvar(_)
+        ; Lval0 = framevar(_)
+        ; Lval0 = parent_stackvar(_)
+        ; Lval0 = succip
+        ; Lval0 = maxfr
+        ; Lval0 = curfr
+        ; Lval0 = hp
+        ; Lval0 = sp
+        ; Lval0 = parent_sp
+        ; Lval0 = lvar(_)
+        ; Lval0 = global_var_ref(_)
+        ),
+        Lval = Lval0
+    ;
+        Lval0 = succip_slot(Rval0),
+        replace_labels_rval(Rval0, Rval, ReplMap),
+        Lval = succip_slot(Rval)
+    ;
+        Lval0 = succfr_slot(Rval0),
+        replace_labels_rval(Rval0, Rval, ReplMap),
+        Lval = succfr_slot(Rval)
+    ;
+        Lval0 = redoip_slot(Rval0),
+        replace_labels_rval(Rval0, Rval, ReplMap),
+        Lval = redoip_slot(Rval)
+    ;
+        Lval0 = redofr_slot(Rval0),
+        replace_labels_rval(Rval0, Rval, ReplMap),
+        Lval = redofr_slot(Rval)
+    ;
+        Lval0 = prevfr_slot(Rval0),
+        replace_labels_rval(Rval0, Rval, ReplMap),
+        Lval = prevfr_slot(Rval)
+    ;
+        Lval0 = field(Tag, BaseRval0, OffsetRval0),
+        replace_labels_rval(BaseRval0, BaseRval, ReplMap),
+        replace_labels_rval(OffsetRval0, OffsetRval, ReplMap),
+        Lval = field(Tag, BaseRval, OffsetRval)
+    ;
+        Lval0 = mem_ref(Rval0),
+        replace_labels_rval(Rval0, Rval, ReplMap),
+        Lval = mem_ref(Rval)
+    ).
+
+:- pred replace_labels_rval(rval::in, rval::out, map(label, label)::in) is det.
+
+replace_labels_rval(Rval0, Rval, ReplMap) :-
+    (
+        Rval0 = lval(Lval0),
+        replace_labels_lval(Lval0, Lval, ReplMap),
+        Rval = lval(Lval)
+    ;
+        Rval0 = var(_Var),
+        Rval = Rval0
+    ;
+        Rval0 = mkword(Tag, SubRval0),
+        replace_labels_rval(SubRval0, SubRval, ReplMap),
+        Rval = mkword(Tag, SubRval)
+    ;
+        Rval0 = const(Const0),
+        replace_labels_rval_const(Const0, Const, ReplMap),
+        Rval = const(Const)
+    ;
+        Rval0 = unop(UnOp, SubRvalA0),
+        replace_labels_rval(SubRvalA0, SubRvalA, ReplMap),
+        Rval = unop(UnOp, SubRvalA)
+    ;
+        Rval0 = binop(BinOp, SubRvalA0, SubRvalB0),
+        replace_labels_rval(SubRvalA0, SubRvalA, ReplMap),
+        replace_labels_rval(SubRvalB0, SubRvalB, ReplMap),
+        Rval = binop(BinOp, SubRvalA, SubRvalB)
+    ;
+        Rval0 = mem_addr(MemRef0),
+        replace_labels_mem_ref(MemRef0, MemRef, ReplMap),
+        Rval = mem_addr(MemRef)
+    ).
+
+:- pred replace_labels_mem_ref(mem_ref::in, mem_ref::out,
+    map(label, label)::in) is det.
+
+replace_labels_mem_ref(MemRef0, MemRef, ReplMap) :-
+    (
+        ( MemRef0 = stackvar_ref(_)
+        ; MemRef0 = framevar_ref(_)
+        ),
+        MemRef = MemRef0
+    ;
+        MemRef0 = heap_ref(CellRval0, Tag, FieldNumRval0),
+        replace_labels_rval(CellRval0, CellRval, ReplMap),
+        replace_labels_rval(FieldNumRval0, FieldNumRval, ReplMap),
+        MemRef = heap_ref(CellRval, Tag, FieldNumRval)
+    ).
+
+:- pred replace_labels_rval_const(rval_const::in, rval_const::out,
+    map(label, label)::in) is det.
+
+replace_labels_rval_const(Const0, Const, ReplMap) :-
+    (
+        ( Const0 = llconst_true
+        ; Const0 = llconst_false
+        ; Const0 = llconst_int(_)
+        ; Const0 = llconst_float(_)
+        ; Const0 = llconst_string(_)
+        ; Const0 = llconst_multi_string(_)
+        ; Const0 = llconst_data_addr(_, _)
+        ),
+        Const = Const0
+    ;
+        Const0 = llconst_code_addr(Addr0),
+        replace_labels_code_addr(Addr0, Addr, ReplMap),
+        Const = llconst_code_addr(Addr)
+    ).
+
+replace_labels_code_addr(Addr0, Addr, ReplMap) :-
+    (
+        Addr0 = code_label(Label0),
+        replace_labels_label(Label0, Label, ReplMap),
+        Addr = code_label(Label)
+    ;
+        ( Addr0 = code_imported_proc(_)
+        ; Addr0 = code_succip
+        ; Addr0 = do_succeed(_)
+        ; Addr0 = do_redo
+        ; Addr0 = do_fail
+        ; Addr0 = do_trace_redo_fail_shallow
+        ; Addr0 = do_trace_redo_fail_deep
+        ; Addr0 = do_call_closure(_)
+        ; Addr0 = do_call_class_method(_)
+        ; Addr0 = do_not_reached
+        ),
+        Addr = Addr0
     ).
 
-replace_labels_comps([], _, []).
-replace_labels_comps([Comp0 | Comps0], ReplMap, [Comp | Comps]) :-
-    replace_labels_comp(Comp0, ReplMap, Comp),
-    replace_labels_comps(Comps0, ReplMap, Comps).
-
-:- pred replace_labels_comp(foreign_proc_component::in,
-    map(label, label)::in, foreign_proc_component::out) is det.
-
-replace_labels_comp(foreign_proc_inputs(A), _, foreign_proc_inputs(A)).
-replace_labels_comp(foreign_proc_outputs(A), _, foreign_proc_outputs(A)).
-replace_labels_comp(foreign_proc_user_code(A, B, C), _,
-        foreign_proc_user_code(A, B, C)).
-replace_labels_comp(foreign_proc_raw_code(A, B, C, D), _,
-        foreign_proc_raw_code(A, B, C, D)).
-replace_labels_comp(foreign_proc_fail_to(Label0), ReplMap,
-        foreign_proc_fail_to(Label)) :-
-    replace_labels_label(Label0, ReplMap, Label).
-replace_labels_comp(foreign_proc_noop, _, foreign_proc_noop).
-
-:- pred replace_labels_lval(lval::in, map(label, label)::in, lval::out) is det.
-
-replace_labels_lval(reg(RegType, RegNum), _, reg(RegType, RegNum)).
-replace_labels_lval(stackvar(N), _, stackvar(N)).
-replace_labels_lval(parent_stackvar(N), _, parent_stackvar(N)).
-replace_labels_lval(framevar(N), _, framevar(N)).
-replace_labels_lval(succip, _, succip).
-replace_labels_lval(maxfr, _, maxfr).
-replace_labels_lval(curfr, _, curfr).
-replace_labels_lval(succip_slot(Rval0), ReplMap, succip_slot(Rval)) :-
-    replace_labels_rval(Rval0, ReplMap, Rval).
-replace_labels_lval(redoip_slot(Rval0), ReplMap, redoip_slot(Rval)) :-
-    replace_labels_rval(Rval0, ReplMap, Rval).
-replace_labels_lval(redofr_slot(Rval0), ReplMap, redofr_slot(Rval)) :-
-    replace_labels_rval(Rval0, ReplMap, Rval).
-replace_labels_lval(succfr_slot(Rval0), ReplMap, succfr_slot(Rval)) :-
-    replace_labels_rval(Rval0, ReplMap, Rval).
-replace_labels_lval(prevfr_slot(Rval0), ReplMap, prevfr_slot(Rval)) :-
-    replace_labels_rval(Rval0, ReplMap, Rval).
-replace_labels_lval(hp, _, hp).
-replace_labels_lval(sp, _, sp).
-replace_labels_lval(parent_sp, _, parent_sp).
-replace_labels_lval(field(Tag, Base0, Offset0), ReplMap,
-        field(Tag, Base, Offset)) :-
-    replace_labels_rval(Base0, ReplMap, Base),
-    replace_labels_rval(Offset0, ReplMap, Offset).
-replace_labels_lval(lvar(Var), _, lvar(Var)).
-replace_labels_lval(temp(Type, Num), _, temp(Type, Num)).
-replace_labels_lval(mem_ref(Rval0), ReplMap, mem_ref(Rval)) :-
-    replace_labels_rval(Rval0, ReplMap, Rval).
-replace_labels_lval(global_var_ref(Name), _, global_var_ref(Name)).
-
-:- pred replace_labels_rval(rval::in, map(label, label)::in,
-    rval::out) is det.
-
-replace_labels_rval(lval(Lval0), ReplMap, lval(Lval)) :-
-    replace_labels_lval(Lval0, ReplMap, Lval).
-replace_labels_rval(var(Var), _, var(Var)).
-replace_labels_rval(mkword(Tag, Rval0), ReplMap,
-        mkword(Tag, Rval)) :-
-    replace_labels_rval(Rval0, ReplMap, Rval).
-replace_labels_rval(const(Const0), ReplMap, const(Const)) :-
-    replace_labels_rval_const(Const0, ReplMap, Const).
-replace_labels_rval(unop(Op, Rval0), ReplMap, unop(Op, Rval)) :-
-    replace_labels_rval(Rval0, ReplMap, Rval).
-replace_labels_rval(binop(Op, LRval0, RRval0), ReplMap,
-        binop(Op, LRval, RRval)) :-
-    replace_labels_rval(LRval0, ReplMap, LRval),
-    replace_labels_rval(RRval0, ReplMap, RRval).
-replace_labels_rval(mem_addr(MemRef0), ReplMap, mem_addr(MemRef)) :-
-    replace_labels_mem_ref(MemRef0, ReplMap, MemRef).
-
-:- pred replace_labels_mem_ref(mem_ref::in, map(label, label)::in,
-    mem_ref::out) is det.
-
-replace_labels_mem_ref(stackvar_ref(N), _, stackvar_ref(N)).
-replace_labels_mem_ref(framevar_ref(N), _, framevar_ref(N)).
-replace_labels_mem_ref(heap_ref(Rval0, Tag, N), ReplMap,
-        heap_ref(Rval, Tag, N)) :-
-    replace_labels_rval(Rval0, ReplMap, Rval).
-
-:- pred replace_labels_rval_const(rval_const::in,
-    map(label, label)::in, rval_const::out) is det.
-
-replace_labels_rval_const(llconst_true, _, llconst_true).
-replace_labels_rval_const(llconst_false, _, llconst_false).
-replace_labels_rval_const(llconst_int(N), _, llconst_int(N)).
-replace_labels_rval_const(llconst_float(N), _, llconst_float(N)).
-replace_labels_rval_const(llconst_string(S), _, llconst_string(S)).
-replace_labels_rval_const(llconst_multi_string(S), _, llconst_multi_string(S)).
-replace_labels_rval_const(llconst_code_addr(Addr0), ReplMap,
-        llconst_code_addr(Addr)) :-
-    replace_labels_code_addr(Addr0, ReplMap, Addr).
-replace_labels_rval_const(llconst_data_addr(DataAddr, MaybeOffset), _,
-        llconst_data_addr(DataAddr, MaybeOffset)).
-
-replace_labels_code_addr(code_label(Label0), ReplMap, code_label(Label)) :-
-    replace_labels_label(Label0, ReplMap, Label).
-replace_labels_code_addr(code_imported_proc(Proc), _,
-        code_imported_proc(Proc)).
-replace_labels_code_addr(code_succip, _, code_succip).
-replace_labels_code_addr(do_succeed(Last), _, do_succeed(Last)).
-replace_labels_code_addr(do_redo, _, do_redo).
-replace_labels_code_addr(do_fail, _, do_fail).
-replace_labels_code_addr(do_trace_redo_fail_shallow, _,
-    do_trace_redo_fail_shallow).
-replace_labels_code_addr(do_trace_redo_fail_deep, _,
-    do_trace_redo_fail_deep).
-replace_labels_code_addr(do_call_closure(MaybeSpec), _,
-        do_call_closure(MaybeSpec)).
-replace_labels_code_addr(do_call_class_method(MaybeSpec), _,
-        do_call_class_method(MaybeSpec)).
-replace_labels_code_addr(do_not_reached, _, do_not_reached).
-
-replace_labels_label_list([], _ReplMap, []).
-replace_labels_label_list([Label0 | Labels0], ReplMap, [Label | Labels]) :-
-    replace_labels_label(Label0, ReplMap, Label),
-    replace_labels_label_list(Labels0, ReplMap, Labels).
+replace_labels_label_list([], [], _ReplMap).
+replace_labels_label_list([Label0 | Labels0], [Label | Labels], ReplMap) :-
+    replace_labels_label(Label0, Label, ReplMap),
+    replace_labels_label_list(Labels0, Labels, ReplMap).
 
-replace_labels_label(Label0, ReplMap, Label) :-
+replace_labels_label(Label0, Label, ReplMap) :-
     ( map.search(ReplMap, Label0, NewLabel) ->
         Label = NewLabel
     ;
Index: compiler/options.m
===================================================================
RCS file: /home/mercury/mercury1/repository/mercury/compiler/options.m,v
retrieving revision 1.579
diff -u -b -r1.579 options.m
--- compiler/options.m	20 Jul 2007 01:22:01 -0000	1.579
+++ compiler/options.m	25 Jul 2007 07:17:08 -0000
@@ -465,6 +465,16 @@
     ;       disable_trail_ops
             % This is used to analyze the performance effects of trailing.
 
+    ;       size_region_ite_fixed
+    ;       size_region_disj_fixed
+    ;       size_region_commit_fixed
+
+    ;       size_region_ite_protect
+    ;       size_region_ite_snapshot
+    ;       size_region_disj_protect
+    ;       size_region_disj_snapshot
+    ;       size_region_commit_entry
+
     % Code generation options
     ;       low_level_debug
     ;       table_debug
@@ -567,6 +577,7 @@
     ;       deforestation_size_threshold
     ;       analyse_trail_usage
     ;       optimize_trail_usage
+    ;       optimize_region_ops
     ;       analyse_mm_tabling
     ;       untuple
     ;       tuple
@@ -1185,7 +1196,16 @@
     disable_minimal_model_stack_copy_cut -  bool(no),
     use_minimal_model_stack_copy_pneg   -   bool(no),
     use_minimal_model_stack_copy_cut    -   bool(no), 
-    disable_trail_ops                   -   bool(no)
+    disable_trail_ops                   -   bool(no),
+    % The size_region_* values are just dummies for now; Quan, please set them.
+    size_region_ite_fixed               -   int(4),
+    size_region_disj_fixed              -   int(4),
+    size_region_commit_fixed            -   int(4),
+    size_region_ite_protect             -   int(0),
+    size_region_ite_snapshot            -   int(0),
+    size_region_disj_protect            -   int(0),
+    size_region_disj_snapshot           -   int(0),
+    size_region_commit_entry            -   int(0)
 ]).
 option_defaults_2(code_gen_option, [
     % Code Generation Options
@@ -1277,6 +1297,7 @@
     analyse_closures                    -   bool(no),
     analyse_trail_usage                 -   bool(no),
     optimize_trail_usage                -   bool(no),
+    optimize_region_ops                 -   bool(no),
     analyse_mm_tabling                  -   bool(no)
 ]).
 option_defaults_2(optimization_option, [
@@ -1955,6 +1976,14 @@
 long_option("disable-mm-pneg",      disable_minimal_model_stack_copy_pneg).
 long_option("disable-mm-cut",       disable_minimal_model_stack_copy_cut).
 long_option("disable-trail-ops",    disable_trail_ops).
+long_option("size-region-ite-fixed",        size_region_ite_fixed).
+long_option("size-region-disj-fixed",       size_region_disj_fixed).
+long_option("size-region-commit-fixed",     size_region_commit_fixed).
+long_option("size-region-ite-protect",      size_region_ite_protect).
+long_option("size-region-ite-snapshot",     size_region_ite_snapshot).
+long_option("size-region-disj-protect",     size_region_disj_protect).
+long_option("size-region-disj-snapshot",    size_region_disj_snapshot).
+long_option("size-region-commit-entry",     size_region_commit_entry).
 
 % code generation options
 long_option("low-level-debug",      low_level_debug).
@@ -2144,6 +2173,7 @@
 long_option("analyse-local-closures",   analyse_closures).
 long_option("analyse-trail-usage",      analyse_trail_usage).
 long_option("optimize-trail-usage",     optimize_trail_usage).
+long_option("optimize-region-ops",  optimize_region_ops).
 long_option("analyse-mm-tabling",       analyse_mm_tabling).
 long_option("untuple",              untuple).
 long_option("tuple",                tuple).
@@ -4031,6 +4061,18 @@
 %       "--delay-partial-instantiations",
 %       "(This option is not for general use.)",
 %       For documentation, see delay_partial_inst.m
+
+        % All these are developer only options.
+%       "(These options are not for general use.)",
+%       For documentation, see runtime/mercury_region.h.
+%       "--size-region-ite-fixed"
+%       "--size-region-disj-fixed"
+%       "--size-region-commit-fixed"
+%       "--size-region-ite-protect"
+%       "--size-region-ite-snapshot"
+%       "--size-region-disj-protect"
+%       "--size-region-disj-snapshot"
+%       "--size-region-commit-entry"
     ]).
 
 :- pred options_help_code_generation(io::di, io::uo) is det.
@@ -4379,13 +4421,18 @@
         "\tprocedures that will not modify the trail.",
         "\tThis information is used to reduce the overhead",
         "\tof trailing.",
-% `--no-optimize-trail-usage' is a developer-only option.  It 
-% is intended for benchmarking the trail usage optimization.
+% `--no-optimize-trail-usage' is a developer-only option.
+% It is intended for the benchmarking of trail usage optimization.
 % Otherwise, it should not be turned off as doing so interferes with
 % the results of the trail usage analysis.
-        %"--no-optimize-trail-usage",
-        %"\tDo not try and restrict trailing to those parts",
-        %"\tof the program that actually use it.",
+        % "--no-optimize-trail-usage",
+        % "\tDo not try and restrict trailing to those parts",
+        % "\tof the program that actually use it.",
+% `--no-optimize-region-ops' is a developer-only option.
+% It is intended for the benchmarking of region ops optimization.
+        % "--no-optimize-region-ops",
+        % "\tDo not try and restrict region operations to those parts",
+        % "\tof the program that actually use it.",
         "--analyse-mm-tabling",
         "\tIdentify those goals that do not call procedures",
         "\tthat are evaluated using minimal model tabling.",
Index: compiler/par_conj_gen.m
===================================================================
RCS file: /home/mercury/mercury1/repository/mercury/compiler/par_conj_gen.m,v
retrieving revision 1.33
diff -u -b -r1.33 par_conj_gen.m
--- compiler/par_conj_gen.m	13 Jul 2007 03:27:27 -0000	1.33
+++ compiler/par_conj_gen.m	23 Jul 2007 15:15:47 -0000
@@ -115,7 +115,6 @@
 :- import_module libs.options.
 :- import_module libs.tree.
 :- import_module ll_backend.code_gen.
-:- import_module ll_backend.code_info.
 :- import_module ll_backend.continuation_info.
 :- import_module ll_backend.exprn_aux.
 :- import_module parse_tree.prog_data.
@@ -151,8 +150,8 @@
     % finishes.
     code_info.get_par_conj_depth(!.CI, Depth),
     ( Depth = 0 ->
-        code_info.acquire_temp_slot(lval(parent_sp), non_persistent_temp_slot,
-            ParentSpSlot, !CI),
+        code_info.acquire_temp_slot(slot_lval(parent_sp),
+            non_persistent_temp_slot, ParentSpSlot, !CI),
         MaybeSetParentSpCode = node([
             llds_instr(assign(ParentSpSlot, lval(parent_sp)),
                 "save the old parent stack pointer"),
@@ -183,7 +182,7 @@
 
     list.length(Goals, NumGoals),
     code_info.acquire_reg(reg_r, RegLval, !CI),
-    code_info.acquire_temp_slot(sync_term, persistent_temp_slot, SyncSlot,
+    code_info.acquire_temp_slot(slot_sync_term, persistent_temp_slot, SyncSlot,
         !CI),
     ( SyncSlot = stackvar(SlotNum) ->
         ParentSyncSlot = parent_stackvar(SlotNum)
Index: compiler/pragma_c_gen.m
===================================================================
RCS file: /home/mercury/mercury1/repository/mercury/compiler/pragma_c_gen.m,v
retrieving revision 1.105
diff -u -b -r1.105 pragma_c_gen.m
--- compiler/pragma_c_gen.m	7 May 2007 05:21:35 -0000	1.105
+++ compiler/pragma_c_gen.m	23 Jul 2007 15:15:47 -0000
@@ -799,8 +799,8 @@
     code_info.maybe_save_hp(ReclaimHeap, SaveHeapCode, MaybeHpSlot, !CI),
     code_info.maybe_restore_hp(MaybeHpSlot, RestoreHeapCode),
 
-    globals.lookup_bool_option(Globals, use_trail, UseTrail),
-    code_info.maybe_save_ticket(UseTrail, SaveTicketCode, MaybeTicketSlot,
+    code_info.get_emit_trail_ops(!.CI, AddTrailOps),
+    code_info.maybe_save_ticket(AddTrailOps, SaveTicketCode, MaybeTicketSlot,
         !CI),
     code_info.maybe_reset_ticket(MaybeTicketSlot, reset_reason_undo,
         RestoreTicketCode),
Index: compiler/proc_gen.m
===================================================================
RCS file: /home/mercury/mercury1/repository/mercury/compiler/proc_gen.m,v
retrieving revision 1.19
diff -u -b -r1.19 proc_gen.m
--- compiler/proc_gen.m	17 May 2007 03:52:50 -0000	1.19
+++ compiler/proc_gen.m	30 Jul 2007 05:19:30 -0000
@@ -769,7 +769,7 @@
                 % Generate code which discards the ticket only if it was
                 % allocated, i.e. only if MR_trace_from_full was true on entry.
                 FromFullSlotLval =
-                    llds.stack_slot_num_to_lval(model_non, FromFullSlot),
+                    llds.stack_slot_num_to_lval(nondet_stack, FromFullSlot),
                 code_info.get_next_label(SkipLabel, !CI),
                 DiscardTraceTicketCode = node([
                     llds_instr(
@@ -1065,9 +1065,10 @@
                 % generate two different copies of this with different labels;
                 % this is needed for semidet code, which will get one copy
                 % in the success epilogue and one copy in the failure epilogue.
-                %
+
+                StackId = code_model_to_main_stack(CodeModel),
                 FromFullSlotLval =
-                    llds.stack_slot_num_to_lval(CodeModel, FromFullSlot),
+                    llds.stack_slot_num_to_lval(StackId, FromFullSlot),
                 code_info.get_next_label(SkipLabel, !CI),
                 code_info.get_next_label(SkipLabelCopy, !CI),
                 PruneTraceTicketCode = node([
Index: compiler/prog_type.m
===================================================================
RCS file: /home/mercury/mercury1/repository/mercury/compiler/prog_type.m,v
retrieving revision 1.36
diff -u -b -r1.36 prog_type.m
--- compiler/prog_type.m	20 Jul 2007 08:09:52 -0000	1.36
+++ compiler/prog_type.m	23 Jul 2007 15:15:47 -0000
@@ -300,6 +300,10 @@
 :- func io_state_type = mer_type.
 :- func region_type = mer_type.
 
+    % Succeed iff the given variable is of region_type.
+    %
+:- pred is_region_var(vartypes::in, prog_var::in) is semidet.
+
     % Construct the types of type_infos and type_ctor_infos.
     %
 :- func type_info_type = mer_type.
@@ -910,6 +914,10 @@
     Module = mercury_region_builtin_module,
     Name = qualified(Module, "region").
 
+is_region_var(VarTypes, Var)  :-
+    map.lookup(VarTypes, Var, Type),
+    Type = region_type.
+
 %-----------------------------------------------------------------------------%
 
     % Given a constant and an arity, return a type_ctor.
Index: compiler/reassign.m
===================================================================
RCS file: /home/mercury/mercury1/repository/mercury/compiler/reassign.m,v
retrieving revision 1.24
diff -u -b -r1.24 reassign.m
--- compiler/reassign.m	9 Jul 2007 13:28:34 -0000	1.24
+++ compiler/reassign.m	30 Jul 2007 05:32:51 -0000
@@ -47,8 +47,8 @@
 % a value that it already holds, we remove the assignment.  The removed
 % assignment will either be a copy of the original assignment TargetLval =
 % SourceRval, or its converse, SourceLval = lval(TargetLval).  The mechanism
-% that enables us to do this is a map that maps lvals
-% (e.g. TargetLval) to its known contents (e.g. SourceRval).
+% that enables us to do this is a map that maps lvals (e.g. TargetLval)
+% to its known contents (e.g. SourceRval).
 %
 % Of course, if any of the lvals occurring on the right hand side of an
 % assignment change, we cannot remove a later copy of that assignment or of
@@ -114,6 +114,7 @@
 :- import_module libs.compiler_util.
 :- import_module ll_backend.code_util.
 
+:- import_module int.
 :- import_module map.
 :- import_module pair.
 :- import_module set.
@@ -171,6 +172,9 @@
             )
         )
     ;
+        Uinstr0 = keep_assign(_, _),
+        !:RevInstrs = [Instr0 | !.RevInstrs]
+    ;
         Uinstr0 = llcall(_, _, _, _, _, _),
         !:RevInstrs = [Instr0 | !.RevInstrs],
         % The call may clobber any lval.
@@ -214,7 +218,8 @@
     ;
         Uinstr0 = save_maxfr(Target),
         !:RevInstrs = [Instr0 | !.RevInstrs],
-        clobber_dependents(Target, !KnownContentsMap, !DepLvalMap)
+        clobber_dependents(Target, !KnownContentsMap, !DepLvalMap),
+        svmap.delete(Target, !KnownContentsMap)
     ;
         Uinstr0 = restore_maxfr(_),
         !:RevInstrs = [Instr0 | !.RevInstrs],
@@ -227,7 +232,8 @@
     ;
         Uinstr0 = mark_hp(Target),
         !:RevInstrs = [Instr0 | !.RevInstrs],
-        clobber_dependents(Target, !KnownContentsMap, !DepLvalMap)
+        clobber_dependents(Target, !KnownContentsMap, !DepLvalMap),
+        svmap.delete(Target, !KnownContentsMap)
     ;
         Uinstr0 = restore_hp(_),
         !:RevInstrs = [Instr0 | !.RevInstrs],
@@ -238,6 +244,21 @@
         % There is no need to update KnownContentsMap since later code
         % should never refer to the freed cell.
     ;
+        ( Uinstr0 = push_region_frame(_, EmbeddedFrame)
+        ; Uinstr0 = region_set_fixed_slot(_, EmbeddedFrame, _)
+        ; Uinstr0 = use_and_maybe_pop_region_frame(_, EmbeddedFrame)
+        ),
+        !:RevInstrs = [Instr0 | !.RevInstrs],
+        update_embdedded_frame(EmbeddedFrame, !KnownContentsMap, !DepLvalMap)
+    ;
+        Uinstr0 = region_fill_frame(_, EmbeddedFrame, _, NumLval, AddrLval),
+        !:RevInstrs = [Instr0 | !.RevInstrs],
+        update_embdedded_frame(EmbeddedFrame, !KnownContentsMap, !DepLvalMap),
+        clobber_dependents(NumLval, !KnownContentsMap, !DepLvalMap),
+        clobber_dependents(AddrLval, !KnownContentsMap, !DepLvalMap),
+        svmap.delete(NumLval, !KnownContentsMap),
+        svmap.delete(AddrLval, !KnownContentsMap)
+    ;
         Uinstr0 = store_ticket(Target),
         !:RevInstrs = [Instr0 | !.RevInstrs],
         clobber_dependents(Target, !KnownContentsMap, !DepLvalMap)
@@ -313,6 +334,31 @@
     remove_reassign_loop(Instrs0, !.KnownContentsMap, !.DepLvalMap,
         !RevInstrs).
 
+:- pred update_embdedded_frame(embedded_stack_frame_id::in,
+    known_contents::in, known_contents::out,
+    dependent_lval_map::in, dependent_lval_map::out) is det.
+
+update_embdedded_frame(EmbeddedFrame, !KnownContentsMap, !DepLvalMap) :-
+    EmbeddedFrame = embedded_stack_frame_id(StackId, FirstSlot, LastSlot),
+    update_embdedded_frame_2(StackId, FirstSlot, LastSlot,
+        !KnownContentsMap, !DepLvalMap).
+
+:- pred update_embdedded_frame_2(main_stack::in, int::in, int::in,
+    known_contents::in, known_contents::out,
+    dependent_lval_map::in, dependent_lval_map::out) is det.
+
+update_embdedded_frame_2(StackId, CurSlot, LastSlot,
+        !KnownContentsMap, !DepLvalMap) :-
+    ( CurSlot =< LastSlot ->
+        StackVar = stack_slot_num_to_lval(StackId, CurSlot),
+        clobber_dependents(StackVar, !KnownContentsMap, !DepLvalMap),
+        svmap.delete(StackVar, !KnownContentsMap),
+        update_embdedded_frame_2(StackId, CurSlot + 1, LastSlot,
+            !KnownContentsMap, !DepLvalMap)
+    ;
+        true
+    ).
+
     % Succeed iff the lval cannot have an alias created for it without the
     % use of a mem_ref lval or an instruction with embedded C code, both of
     % which cause us to clobber the known contents map.
@@ -342,8 +388,10 @@
     % that may be referred to via this mem_ref.
     code_util.lvals_in_rval(lval(Target), SubLvals),
     (
+        some [SubLval] (
         list.member(SubLval, SubLvals),
         SubLval = mem_ref(_)
+        )
     ->
         !:KnownContentsMap = map.init,
         !:DepLvalMap = map.init
Index: compiler/stack_layout.m
===================================================================
RCS file: /home/mercury/mercury1/repository/mercury/compiler/stack_layout.m,v
retrieving revision 1.135
diff -u -b -r1.135 stack_layout.m
--- compiler/stack_layout.m	12 Jun 2007 06:06:30 -0000	1.135
+++ compiler/stack_layout.m	30 Jul 2007 05:34:58 -0000
@@ -1456,6 +1456,14 @@
     represent_special_live_value_type("trail_ptr", Rval).
 represent_live_value_type(live_value_ticket, Rval, data_ptr, !Info) :-
     represent_special_live_value_type("ticket", Rval).
+represent_live_value_type(RegionType, Rval, data_ptr, !Info) :-
+    ( RegionType = live_value_region_ite
+    ; RegionType = live_value_region_disj
+    ; RegionType = live_value_region_commit
+    ),
+    % Neither the garbage collector nor the debugger need info about
+    % regions.
+    represent_special_live_value_type("unwanted", Rval).
 represent_live_value_type(live_value_unwanted, Rval, data_ptr, !Info) :-
     represent_special_live_value_type("unwanted", Rval).
 represent_live_value_type(live_value_var(_, _, Type, _), Rval, LldsType,
Index: compiler/stdlabel.m
===================================================================
RCS file: /home/mercury/mercury1/repository/mercury/compiler/stdlabel.m,v
retrieving revision 1.4
diff -u -b -r1.4 stdlabel.m
--- compiler/stdlabel.m	6 Jan 2007 09:23:52 -0000	1.4
+++ compiler/stdlabel.m	23 Jul 2007 15:15:47 -0000
@@ -59,7 +59,7 @@
     ->
         build_std_map(Instrs1, ProcLabel, counter.init(1), !:ProcCounter,
             map.init, Map),
-        replace_labels_instruction_list(Instrs1, Map, yes, yes, Instrs2),
+        replace_labels_instruction_list(Instrs1, Instrs2, Map, yes, yes),
         Instrs = [LabelInstr | Comments] ++ Instrs2
     ;
         unexpected(this_file, "standardize_labels: no proc_label")
Index: compiler/trace_gen.m
===================================================================
RCS file: /home/mercury/mercury1/repository/mercury/compiler/trace_gen.m,v
retrieving revision 1.17
diff -u -b -r1.17 trace_gen.m
--- compiler/trace_gen.m	2 May 2007 02:15:59 -0000	1.17
+++ compiler/trace_gen.m	30 Jul 2007 05:36:26 -0000
@@ -476,6 +476,7 @@
     ),
     FromFullSlot = eff_trace_level_needs_from_full_slot(ModuleInfo, PredInfo,
         ProcInfo, TraceLevel),
+    StackId = code_model_to_main_stack(CodeModel),
     (
         FromFullSlot = no,
         MaybeFromFullSlot = no,
@@ -484,16 +485,15 @@
     ;
         FromFullSlot = yes,
         MaybeFromFullSlot = yes(NextSlotAfterRedoLayout),
-        CallFromFullSlot = llds.stack_slot_num_to_lval(
-            CodeModel, NextSlotAfterRedoLayout),
+        CallFromFullSlot = stack_slot_num_to_lval(StackId,
+            NextSlotAfterRedoLayout),
         MaybeFromFullSlotLval = yes(CallFromFullSlot),
         NextSlotAfterFromFull = NextSlotAfterRedoLayout + 1
     ),
     (
         TraceTableIo = yes,
         MaybeIoSeqSlot = yes(NextSlotAfterFromFull),
-        IoSeqLval = llds.stack_slot_num_to_lval(CodeModel,
-            NextSlotAfterFromFull),
+        IoSeqLval = stack_slot_num_to_lval(StackId, NextSlotAfterFromFull),
         MaybeIoSeqLval = yes(IoSeqLval),
         NextSlotAfterIoSeq = NextSlotAfterFromFull + 1
     ;
@@ -506,10 +506,8 @@
     (
         UseTrail = yes,
         MaybeTrailSlot = yes(NextSlotAfterIoSeq),
-        TrailLval = llds.stack_slot_num_to_lval(CodeModel,
-            NextSlotAfterIoSeq),
-        TicketLval = llds.stack_slot_num_to_lval(CodeModel,
-            NextSlotAfterIoSeq + 1),
+        TrailLval = stack_slot_num_to_lval(StackId, NextSlotAfterIoSeq),
+        TicketLval = stack_slot_num_to_lval(StackId, NextSlotAfterIoSeq + 1),
         MaybeTrailLvals = yes(TrailLval - TicketLval),
         NextSlotAfterTrail = NextSlotAfterIoSeq + 2
     ;
@@ -522,8 +520,7 @@
     (
         NeedMaxfr = yes,
         MaybeMaxfrSlot = yes(NextSlotAfterTrail),
-        MaxfrLval = llds.stack_slot_num_to_lval(CodeModel,
-            NextSlotAfterTrail),
+        MaxfrLval = stack_slot_num_to_lval(StackId, NextSlotAfterTrail),
         MaybeMaxfrLval = yes(MaxfrLval),
         NextSlotAfterMaxfr = NextSlotAfterTrail + 1
     ;
@@ -534,8 +531,7 @@
     ),
     ( proc_info_get_call_table_tip(ProcInfo, yes(_)) ->
         MaybeCallTableSlot = yes(NextSlotAfterMaxfr),
-        CallTableLval = llds.stack_slot_num_to_lval(CodeModel,
-            NextSlotAfterMaxfr),
+        CallTableLval = stack_slot_num_to_lval(StackId, NextSlotAfterMaxfr),
         MaybeCallTableLval = yes(CallTableLval)
     ;
         MaybeCallTableSlot = no,
Index: compiler/use_local_vars.m
===================================================================
RCS file: /home/mercury/mercury1/repository/mercury/compiler/use_local_vars.m,v
retrieving revision 1.34
diff -u -b -r1.34 use_local_vars.m
--- compiler/use_local_vars.m	9 Jul 2007 13:28:34 -0000	1.34
+++ compiler/use_local_vars.m	30 Jul 2007 05:39:30 -0000
@@ -173,6 +173,7 @@
     Instr0 = llds_instr(Uinstr0, _Comment0),
     (
         (
+            % We don't optimize keep_assign instructions.
             (
                 Uinstr0 = assign(ToLval, _FromRval)
             ;
@@ -590,7 +591,13 @@
             substitute_lval_in_instr_until_defn(OldLval, NewLval, !Instrs, !N)
         )
     ;
-        Uinstr0 = incr_hp(Lval, _, _, _, _, _, _),
+        Uinstr0 = keep_assign(_, _),
+        exprn_aux.substitute_lval_in_instr(OldLval, NewLval, !Instr, !N)
+    ;
+        ( Uinstr0 = incr_hp(Lval, _, _, _, _, _, _)
+        ; Uinstr0 = save_maxfr(Lval)
+        ; Uinstr0 = mark_hp(Lval)
+        ),
         ( assignment_updates_oldlval(Lval, OldLval) = yes ->
             % If we alter any lval that occurs in OldLval, we must stop
             % the substitutions.
@@ -600,6 +607,29 @@
             substitute_lval_in_instr_until_defn(OldLval, NewLval, !Instrs, !N)
         )
     ;
+        Uinstr0 = region_fill_frame(_, _, _, NumLval, AddrLval),
+        (
+            ( assignment_updates_oldlval(NumLval, OldLval) = yes
+            ; assignment_updates_oldlval(AddrLval, OldLval) = yes
+            )
+        ->
+            % If we alter any lval that occurs in NumLval or AddrLval,
+            % we must stop the substitutions.
+            true
+        ;
+            exprn_aux.substitute_lval_in_instr(OldLval, NewLval, !Instr, !N),
+            substitute_lval_in_instr_until_defn(OldLval, NewLval, !Instrs, !N)
+        )
+    ;
+        ( Uinstr0 = restore_maxfr(_)
+        ; Uinstr0 = restore_hp(_)
+        ; Uinstr0 = push_region_frame(_, _)
+        ; Uinstr0 = region_set_fixed_slot(_, _, _)
+        ; Uinstr0 = use_and_maybe_pop_region_frame(_, _)
+        ),
+        exprn_aux.substitute_lval_in_instr(OldLval, NewLval, !Instr, !N),
+        substitute_lval_in_instr_until_defn(OldLval, NewLval, !Instrs, !N)
+    ;
         Uinstr0 = foreign_proc_code(_, Components, _, _, _, _, _, _, _),
         AffectsLiveness = components_affect_liveness(Components),
         (
@@ -633,10 +663,6 @@
         ; Uinstr0 = llcall(_, _, _, _, _, _)
         ; Uinstr0 = mkframe(_, _)
         ; Uinstr0 = goto(_)
-        ; Uinstr0 = save_maxfr(_)
-        ; Uinstr0 = restore_maxfr(_)
-        ; Uinstr0 = mark_hp(_)
-        ; Uinstr0 = restore_hp(_)
         ; Uinstr0 = free_heap(_)
         ; Uinstr0 = store_ticket(_)
         ; Uinstr0 = reset_ticket(_, _)
cvs diff: Diffing compiler/notes
cvs diff: Diffing debian
cvs diff: Diffing debian/patches
cvs diff: Diffing deep_profiler
cvs diff: Diffing deep_profiler/notes
cvs diff: Diffing doc
cvs diff: Diffing extras
cvs diff: Diffing extras/base64
cvs diff: Diffing extras/cgi
cvs diff: Diffing extras/complex_numbers
cvs diff: Diffing extras/complex_numbers/samples
cvs diff: Diffing extras/complex_numbers/tests
cvs diff: Diffing extras/concurrency
cvs diff: Diffing extras/curs
cvs diff: Diffing extras/curs/samples
cvs diff: Diffing extras/curses
cvs diff: Diffing extras/curses/sample
cvs diff: Diffing extras/dynamic_linking
cvs diff: Diffing extras/error
cvs diff: Diffing extras/fixed
cvs diff: Diffing extras/gator
cvs diff: Diffing extras/gator/generations
cvs diff: Diffing extras/gator/generations/1
cvs diff: Diffing extras/graphics
cvs diff: Diffing extras/graphics/easyx
cvs diff: Diffing extras/graphics/easyx/samples
cvs diff: Diffing extras/graphics/mercury_allegro
cvs diff: Diffing extras/graphics/mercury_allegro/examples
cvs diff: Diffing extras/graphics/mercury_allegro/samples
cvs diff: Diffing extras/graphics/mercury_allegro/samples/demo
cvs diff: Diffing extras/graphics/mercury_allegro/samples/mandel
cvs diff: Diffing extras/graphics/mercury_allegro/samples/pendulum2
cvs diff: Diffing extras/graphics/mercury_allegro/samples/speed
cvs diff: Diffing extras/graphics/mercury_glut
cvs diff: Diffing extras/graphics/mercury_opengl
cvs diff: Diffing extras/graphics/mercury_tcltk
cvs diff: Diffing extras/graphics/samples
cvs diff: Diffing extras/graphics/samples/calc
cvs diff: Diffing extras/graphics/samples/gears
cvs diff: Diffing extras/graphics/samples/maze
cvs diff: Diffing extras/graphics/samples/pent
cvs diff: Diffing extras/lazy_evaluation
cvs diff: Diffing extras/lex
cvs diff: Diffing extras/lex/samples
cvs diff: Diffing extras/lex/tests
cvs diff: Diffing extras/log4m
cvs diff: Diffing extras/logged_output
cvs diff: Diffing extras/moose
cvs diff: Diffing extras/moose/samples
cvs diff: Diffing extras/moose/tests
cvs diff: Diffing extras/mopenssl
cvs diff: Diffing extras/morphine
cvs diff: Diffing extras/morphine/non-regression-tests
cvs diff: Diffing extras/morphine/scripts
cvs diff: Diffing extras/morphine/source
cvs diff: Diffing extras/net
cvs diff: Diffing extras/odbc
cvs diff: Diffing extras/posix
cvs diff: Diffing extras/posix/samples
cvs diff: Diffing extras/quickcheck
cvs diff: Diffing extras/quickcheck/tutes
cvs diff: Diffing extras/references
cvs diff: Diffing extras/references/samples
cvs diff: Diffing extras/references/tests
cvs diff: Diffing extras/solver_types
cvs diff: Diffing extras/solver_types/library
cvs diff: Diffing extras/trailed_update
cvs diff: Diffing extras/trailed_update/samples
cvs diff: Diffing extras/trailed_update/tests
cvs diff: Diffing extras/windows_installer_generator
cvs diff: Diffing extras/windows_installer_generator/sample
cvs diff: Diffing extras/windows_installer_generator/sample/images
cvs diff: Diffing extras/xml
cvs diff: Diffing extras/xml/samples
cvs diff: Diffing extras/xml_stylesheets
cvs diff: Diffing java
cvs diff: Diffing java/runtime
cvs diff: Diffing library
cvs diff: Diffing mdbcomp
cvs diff: Diffing profiler
cvs diff: Diffing robdd
cvs diff: Diffing runtime
cvs diff: Diffing runtime/GETOPT
cvs diff: Diffing runtime/machdeps
cvs diff: Diffing samples
cvs diff: Diffing samples/c_interface
cvs diff: Diffing samples/c_interface/c_calls_mercury
cvs diff: Diffing samples/c_interface/cplusplus_calls_mercury
cvs diff: Diffing samples/c_interface/mercury_calls_c
cvs diff: Diffing samples/c_interface/mercury_calls_cplusplus
cvs diff: Diffing samples/c_interface/mercury_calls_fortran
cvs diff: Diffing samples/c_interface/simpler_c_calls_mercury
cvs diff: Diffing samples/c_interface/simpler_cplusplus_calls_mercury
cvs diff: Diffing samples/c_interface/standalone_c
cvs diff: Diffing samples/diff
cvs diff: Diffing samples/muz
cvs diff: Diffing samples/rot13
cvs diff: Diffing samples/solutions
cvs diff: Diffing samples/solver_types
cvs diff: Diffing samples/tests
cvs diff: Diffing samples/tests/c_interface
cvs diff: Diffing samples/tests/c_interface/c_calls_mercury
cvs diff: Diffing samples/tests/c_interface/cplusplus_calls_mercury
cvs diff: Diffing samples/tests/c_interface/mercury_calls_c
cvs diff: Diffing samples/tests/c_interface/mercury_calls_cplusplus
cvs diff: Diffing samples/tests/c_interface/mercury_calls_fortran
cvs diff: Diffing samples/tests/c_interface/simpler_c_calls_mercury
cvs diff: Diffing samples/tests/c_interface/simpler_cplusplus_calls_mercury
cvs diff: Diffing samples/tests/diff
cvs diff: Diffing samples/tests/muz
cvs diff: Diffing samples/tests/rot13
cvs diff: Diffing samples/tests/solutions
cvs diff: Diffing samples/tests/toplevel
cvs diff: Diffing scripts
cvs diff: Diffing slice
cvs diff: Diffing tests
cvs diff: Diffing tests/benchmarks
cvs diff: Diffing tests/debugger
cvs diff: Diffing tests/debugger/declarative
cvs diff: Diffing tests/dppd
cvs diff: Diffing tests/general
cvs diff: Diffing tests/general/accumulator
cvs diff: Diffing tests/general/string_format
cvs diff: Diffing tests/general/structure_reuse
cvs diff: Diffing tests/grade_subdirs
cvs diff: Diffing tests/hard_coded
cvs diff: Diffing tests/hard_coded/exceptions
cvs diff: Diffing tests/hard_coded/purity
cvs diff: Diffing tests/hard_coded/sub-modules
cvs diff: Diffing tests/hard_coded/typeclasses
cvs diff: Diffing tests/invalid
cvs diff: Diffing tests/invalid/purity
cvs diff: Diffing tests/misc_tests
cvs diff: Diffing tests/mmc_make
cvs diff: Diffing tests/mmc_make/lib
cvs diff: Diffing tests/par_conj
cvs diff: Diffing tests/recompilation
cvs diff: Diffing tests/tabling
cvs diff: Diffing tests/term
cvs diff: Diffing tests/trailing
cvs diff: Diffing tests/valid
cvs diff: Diffing tests/warnings
cvs diff: Diffing tools
cvs diff: Diffing trace
cvs diff: Diffing util
cvs diff: Diffing vim
cvs diff: Diffing vim/after
cvs diff: Diffing vim/ftplugin
cvs diff: Diffing vim/syntax
--------------------------------------------------------------------------
mercury-reviews mailing list
Post messages to:       mercury-reviews at csse.unimelb.edu.au
Administrative Queries: owner-mercury-reviews at csse.unimelb.edu.au
Subscriptions:          mercury-reviews-request at csse.unimelb.edu.au
--------------------------------------------------------------------------



More information about the reviews mailing list