[m-dev.] for review: the new debugger command set (part 1 of 5)

[Zoltan Somogyi]

> And this is entirely the sort of information that is lost when you
> commit a bunch of changes like this together.  Anyone who want  to
> see how a change works or doesn't work, or how to merge it successfully,
> or how to back it out will need to reconstruct the dependencies like
> this.  
> 
> I have no problem with dependent or releated changes being checked in
> together.
> 
> Documentation, redo events, variable identification, and bug fixes

The bug fixes, at least some of them, are too entangled with the rest
of the code to tease out, without redoing them alomost from scratch.
I know, because I just spent several hours trying to separate out some
changes (starting numbering framevars from 1 and fixing the bug with
want_return_layout) from the diff, without much success, because
often nearby lines of code are affected by different changes, and
patch doesn't work very well on hand-edited context diffs.
This is unavoidable, because the bug fixes and enhancements you want me
to separate out are integral to the main point of the diff, which is the
implementation of the new debugger command set, and having the compiler
make available the information this requires. When you discover the need
for a bug fix or enhancement after having accumulated some diffs,
you have two unpalatable choices: fix it only in your current workspace,
which links the two changes, or fix it also in a fresh workspace, which
doubles your work.

> and I don't want to encourage a culture of working for 3 months in
> a single workspace and then committing it all as one big change.
> Even if it was unavoidable, or justifiable, I'd just rather it didn't
> happen.

Nor would I want to encourage such a culture. But sometimes it is
the lesser of two evils.

> A compromise will be fine.  Clearly we have both indicated that this is
> too much at once -- you could probably have halved it into
> "documentation" and "everything else" and got it past without comment.

Huh? I *tried* that, and it got knocked back because "it wasn't documented".

> So perhaps we can split off the "redo event" change from the online
> documentation change at least, and then work on getting them commited
> separately.

The documentation stuff is mostly add-on, and it can be committed separately.
The problem is, whichever order the commits take place in, for some time
the code and the doc will be inconsistent.

If I were to try to divide this diff into several significantly smaller parts,
it would take me another two weeks of work (taking probably two months of
calendar time due to the need to bootcheck each change before building on it).
This is not acceptable.

I propose the following.

1. I will fix the problem of the runtime depending on the standard library
by creating two new libraries, corresponding to (a) the call-tree of MR_trace
in the runtime and (b) the call-tree of MR_trace in the library (help.m,
any browsing code we add later, and maybe debugger_interface.m).

2. I will remove from the diff whatever changes can be removed without
excessive work duplication, for separate review later.

3. I will give names to each item in my log message (which correspond, more
or less, with the 17 items Tyson identified), and annotate each component of
the diff with the name(s) of the item(s) that the diff implements. This way,
a reviewer can check whether an item is implemented properly by looking at
only the differences tagged by the name of the item. For a reviewer, the
effect should be quite similar to reviewing a sequence of smaller diffs,
with the only difference being the requirement to be aware of other,
interacting changes.

4. To supply this awareness if you don't have it, I can conduct a walkthrough
of the annotated diff for the reviewers. I nominate thursday afternoon.

Zoltan.