[m-dev.] [Gc] Article about memory fragmentation

[Bruce Hoult]

Good article, as far as it goes, but it just .. stops .. before the
interesting analysis.

Btw "uncorrectable" should be "uncollectable".

Almost all those block sizes have excellent utilization ratios. A moving
collector could scarcely do better!

But there's definitely room for improvement with the 32 byte size.

I'm very curious what the memory allocation pattern in that makes you end
up with so many blocks with so few live objects.

My best guess is that at some point you actually did have 60 MB of 32 byte
objects, but then most of them became unreachable. But you're the one who
can gather the object lifetime data.

In a long-running program, you'd expect that space to get used again,
multiple times, so it's not really a problem.

But it can also be that there is a usage peak for some object size during
program startup, and then they are not needed again. Unfortunately, there's
very little that a non-moving GC can do about that.

It might be possible to do something about that at a user level though: if
there are interspersed allocations of short lived and long lived (or
pemanent) objects of the same size, then you could pre-allocate a number of
objects (4 KB worth or some multiple) and put them in your own freelist or
some other collection, and then replace the current GC_malloc of them with
taking one from the pre-allocated list. You could do this for either the
short-lived or the long-lived objects, but I suspect that the long-lived
ones might give the easiest benefit. Get them all onto the same memory
pages, that will stay reasonably full, and let entire pages of short-lived
objects get GC'd and those pages reused for objects of other sizes.

The GC can't be some kind of oracle and predict these usage patterns, but
you might be able to.

On Sun, Oct 9, 2016 at 12:32 AM, Paul Bone <paul at bone.id.au> wrote:

>
> This is a cross post as both communities will probably be interested.
>
> I investigated an issue with memory fragmentation in BDW GC and Mercury and
> it proved to be quite interesting, so I wrote it up as a blog post.
>
> http://paul.bone.id.au/2016/10/08/memory-fragmentation-in-boehmgc/
>
> I also learnt a fair amount about the collector in the process, and
> explained that in the article too.  It may be interesting if you're curious
> about how allocation works in the collector.
>
> Please let me know if I've gotten anything wrong, or of course if you have
> any other comments.
>
> Cheers.
>
>
> --
> Paul Bone
> http://paul.bone.id.au
> _______________________________________________
> bdwgc mailing list
> bdwgc at lists.opendylan.org
> https://lists.opendylan.org/mailman/listinfo/bdwgc
>
> --
> This message has been scanned for viruses and
> dangerous content by MailScanner, and is
> believed to be clean.
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.mercurylang.org/archives/developers/attachments/20161009/268f3a3a/attachment.html>