[m-rev.] for review: Clarify some code relating to redzones.

[Zoltan Somogyi]

On Sun, 1 Nov 2026 00:03:34 +1100, Julien Fischer <jfischer at opturion.com> wrote:

> Hi Zoltan,
> 
> On Sat, 31 Oct 2026 at 20:47, Zoltan Somogyi <zoltan.somogyi at runbox.com> wrote:
> >
> > On Sat, 31 Oct 2026 08:56:13 +1100 (AEDT), "Zoltan Somogyi" <zoltan.somogyi at runbox.com> wrote:
> > > For me, this just replaces one pair of confusing names with another.
> > > The name MR_USE_PAGE_PROTECTION is not all that useful
> > > because it does say *what purpose* we should use page protection for.
> > > And MR_USE_REDZONES is not clear because it assumes that the
> > > reader knows what redzones ARE, and earlier we established that
> > > there are two possible meanings: the last page of a zone that is never
> > > unprotected (provided the platform can protect pages), or a larger
> > > set of pages, some of which can become unprotected.
> >
> > Upon further reflection, I propose the following definition,
> > which I think you are already using, but is not yet explicitly written down,
> > and is not (yet) reflected in structure field names and variable names.
> >
> > - The redzone is the area of memory at the end of a memory zone
> >   (det/nondet stack, heap, etc) that we ever explicitly deny access to,
> >   via mprotect or similar functionality. Both it, and its components
> >   the hard and soft redzones, must have sizes that are exact multiples
> >   of the page size, and their address must be page aligned,
> >   which must be ensured by us, because we cannot rely Boehm doing it.
> >
> > - The hard redzone is the last page of the redzone, which we never
> >   unprotect.
> 
> Right; we also refer to this using the term "hardzone" in a few spots.

We do. I am proposing that we stop doing that, and start using "hard redzone"
instead.

> > - The soft redzone is the rest of the redzone. It starts out with no access
> >   being permitted (or as close to it as mprotect/equivalents can get),
> >   but we undo this protection on access.
> >
> > - The purpose of the hard redzone is definitely to guarantee that we catch
> >   about-to-happen overflows.
> 
> This suggests that for some uses of memory zones we don't need hard redzones.
> (The zones used for stack and trail segments are the obvious examples.)

Exactly. I don't think this interaction was thought through when we introduced
segments.

> > This requires the hard rezone to be at least as big
> >  as the largest stack frame.
> >
> > - The possible purposes of the soft redzone are: (a) keeping track of how much
> >   of a memory zone is actually used, both for plain curiosity and to inform
> >   decisions about how big we should make memory zones, and (b) to help
> >   Boehm gc do its job faster, by ... (fill in the blank, because I don't know
> >   the mechanism by which this would work).
> >   I don't think (a) is relevant anymore, because we can get the same info
> >   in a more useful manner using stack segments. I have no idea whether
> >   (b) is still relevant. If it is not, then we can stop using soft redzones,
> >   though keeping its code around wouldn't hurt, *provided* that the code
> >   is *documented* as being unused.
> 
> I don't think (b) is a thing.  MR_default_handler() is the only function that
> handles access in redzones.  The only interaction it has with a garbage
> collector is in low-level agc grades, where it calls MR_schedule_agc().
> (Presumably, that only really matters when the zone in question is a heap
> zone.)

And also presumably, it only matters if agc works in LLDS grades,
which I do not remember ever happening. Even in MLDS grades,
agc worked only for (subjectively) five minutes.

That is an argument for soft redzones not being needed, and for documenting
them as not being needed. We may still want to keep the trivial amount of code
supporting them, but if we do, we should add comments saying (a) what functionality
they are intended to support, and (b) that there is nothing to support just yet.

> > Do you guys agree?
> 
> In principle, sure. I would also offer the following observation:  using names
> like  "redzone" and "hardzone" inside something called a "memory zone" is
> fairly confusing.

What terminology would you suggest instead?

Zoltan.