[mercury-users] Debugging 'memory leaks' (fwd)

[Julien Fischer]

The following is forwarded from Ondrej (who seems to be
having trouble with the mailing lists at the moment).

-------- Original Message --------
Subject: Debugging 'memory leaks'
Date: Tue, 25 Mar 2008 16:25:39 +0100
From: Ondrej Bojar <bojar at ufal.mff.cuni.cz>
To: mercury-users at cs.mu.oz.au

Hi, all.

My Mercury code is kind of greedy and does not seem to de-allocate all
it should. Parts of the code are in C but I use always
MR_GC_NEW/NEW_ARRAY to allocate. The main looks like this:

main(!IO) :-
   do_something_big(!IO),
   print_memusage(!IO),
   gc.garbage_collect(!IO),
   print_memusage(!IO).

The do_something_big(io::di, io::uo) contains a 'cached' call. From some
hardwired input files, it collects some statistics and saves them to a
cache (and say prints them to stdout). Next time when I run the program,
it just loads the cache and prints the statistics, so stdout looks
identical but the collection was not performed.

There are two things I don't like:

- memory usage as reported by the first print_memusage is drastically
different in the two runs (cached vs. not-yet-cached), do_something_big
allocates some temporary structures when doing the statistics but does
not release it properly, although nothing is passed back to main (except
for altered IO).
- print_memusage shows exactly the same amount of memory used before and
after garbage_collection in both runs, ie. what was left in the memory
still looks reachable for the garbage collector (or gc.garbage_collect
is just a NOOP).

Is there a way to diagnose what is being left in the memory (or when was
it allocated)?

I tried to use valgrind but it did not find any leak: (All the errors
are reported for GC_something.)

Moreover, the number of allocations valgrind found is by several orders
of magnitude lower than the number of calls to MR_GC_NEW I actually
make. (This can be explained by some memory pool in gc, if there is any.)

The two non-freed blocks found are insignificant, the program actually
allocated 10% of RAM on a 16GB machine.

==1018== ERROR SUMMARY: 23773 errors from 41 contexts (suppressed: 5 from 1)
==1018== malloc/free: in use at exit: 1,136 bytes in 2 blocks.
==1018== malloc/free: 1,703 allocs, 1,701 frees, 1,568,672 bytes allocated.
==1018== For counts of detected errors, rerun with: -v
==1018== searching for pointers to 2 not-freed blocks.
==1018== checked 1,175,085,728 bytes.
==1018==
==1018==
==1018== 568 bytes in 1 blocks are still reachable in loss record 1 of 2
==1018==    at 0x4A05996: malloc (vg_replace_malloc.c:149)
==1018==    by 0x3A6445F408: fdopen@@GLIBC_2.2.5 (in /lib64/libc-2.6.so)
==1018==    by 0x50129CE: mercury_init_io (in
/export/work/people/bojar/opt/mercury-compiler-rotd-2007-08-13/lib/mercury/lib/asm_fast.gc/libmer_std.so)
==1018==    by 0x57A6E20: mercury_runtime_init (mercury_wrapper.c:557)
==1018==    by 0x419460: mercury_main (testhugemap_init.c:2951)
==1018==    by 0x3A6441DAB3: (below main) (in /lib64/libc-2.6.so)
==1018==
==1018==
==1018== 568 bytes in 1 blocks are still reachable in loss record 2 of 2
==1018==    at 0x4A05996: malloc (vg_replace_malloc.c:149)
==1018==    by 0x3A6445F408: fdopen@@GLIBC_2.2.5 (in /lib64/libc-2.6.so)
==1018==    by 0x50129A2: mercury_init_io (in
/export/work/people/bojar/opt/mercury-compiler-rotd-2007-08-13/lib/mercury/lib/asm_fast.gc/libmer_std.so)
==1018==    by 0x57A6E20: mercury_runtime_init (mercury_wrapper.c:557)
==1018==    by 0x419460: mercury_main (testhugemap_init.c:2951)
==1018==    by 0x3A6441DAB3: (below main) (in /lib64/libc-2.6.so)

Any hints?

Thanks, Ondrej.

-- 
Ondrej Bojar (mailto:obo at cuni.cz / bojar at ufal.mff.cuni.cz)
http://www.cuni.cz/~obo
--------------------------------------------------------------------------
mercury-users mailing list
Post messages to:       mercury-users at csse.unimelb.edu.au
Administrative Queries: owner-mercury-users at csse.unimelb.edu.au
Subscriptions:          mercury-users-request at csse.unimelb.edu.au
--------------------------------------------------------------------------