[mercury-users] Memory allocation in foreign code in parallel grades

[Christoph Schwering]

Hi everyone,

I'm using Mercury to implement an interpreter for an action language
(a dialect of Golog [1]). Most interpreters are written in Prolog, but
Mercury seems to me to be a great alternative.

My question, in short, is how one can and should allocate memory in
foreign C and C++ code when using a parallel grade: with malloc,
GC_MALLOC, or something else?


The foreign code I'd like to interface is the COIN-OR CLP solver [2]
through the OSI interface [3]. I don't need solver_types. It's very
simple:
* there's a foreign_type defined as SolverContext* where SolverContext
  is a C-typedef,
* there's a pair of foreign_procs new_solver_context and
  finalize_solver_context that creates and destroys a SolverContext
  object with C++ new and delete,
* a foreign_proc add_constraint which adds a new constraint to the
  solver context, and
* a foreign_proc solve.
The attributes of the foreign_procs are will_not_call_mercury,
promise_pure, thread_safe.
Since COIN-OR CLP is written in C++, the underlying C/C++ procedures
use new/delete and malloc/free.
Besides the SolverContext* pointer, the other data passed between
Mercury and C++ are arrays of floats and ints. I copy them, i.e., I
copy Mercury lists of floats/ints to newly malloced arrays of
floats/ints which I pass to C++ and the other way around.

Using the non-parallel grade hlc.gc this worked fine. But when I use
grade hlc.par.gc and create multiple threads in Mercury (COIN-OR CLP
is still single-threaded), I get segmentation fault (SIGSEGV). It
often occurs in GC_build_fl called by GC_generic_malloc_many called by
GC_malloc called by Mercury code in libpar_gc.so, but sometimes it
occurs in other GC functions.


For some reason the following steps fix the problem:
* replace malloc, free, etc. with GC_MALLOC, GC_FREE, etc. in the
  whole COIN-OR code,
* override the new and delete globally to refer to GC_MALLOC and
  GC_FREE, and
* re-compile Mercury's boehm_gc and replace
  lib/mercury/lib/libpar_gc.so with the newly compiled libgc.so (I'm not
  sure whether or not this step is necessary; I forgot why I did it, but
  now it seems to work without it).

Did I do the right thing? If so, can anybody explain to me why?
Doesn't Boehm GC coexist peacefully with system malloc in presence of
threads?
I would very much like to avoid using GC in COIN-OR, because (unless a
SIGSEGV occurs) the non-GC version is about four times as fast as the
GC version.


I saw a comment in the Mercury extras' curs library:
/*
** The addresses of the closures that we pass to curses
** will be stored by curses in malloc()'ed memory.
** However, it is essential that these pointers be
** visible to the garbage collector, otherwise it will
** think that the closures are unreferenced and reuse the storage.
** Hence we redefine malloc() and friends to call GC_malloc().
*/
I guess this has similar effect like my replacing all malloc, free
etc. with GC_MALLOC, GC_FREE etc.
But redefining malloc and friends should only be needed when
Mercury-allocated (and thus GC-allocated) memory finds its way to the
C-code, right? And this isn't the case in my code because the
foreign_procs copy Mercury-lists to newly malloced C-arrays.


I'm using Mercury 11.07.1 under Linux x86_64, GCC 4.6.2 20111027 (Red
Hat 4.6.2-1), and grade hlc.par.gc.

With asm_fast.par.gc the memory consumption is very high and I get
segmentation faults even with one thread. The error message suggests a
stack overflow due to recursion as reason. This really might be the
case, but I wonder why this doesn't occur in non-parallel asm_fast.gc?

And unfortunately, I can't compile the agc grades; the error message
is that control flow reaches end of non-void functions in array.m. Is
this a known issue?

But besides this problem I really like Mercury .

Cheers
Chris

[1] http://www.cs.toronto.edu/cogrobo/main/systems/index.html
[2] https://projects.coin-or.org/Clp/
[3] https://projects.coin-or.org/Osi/

--------------------------------------------------------------------------
mercury-users mailing list
Post messages to:       mercury-users at csse.unimelb.edu.au
Administrative Queries: owner-mercury-users at csse.unimelb.edu.au
Subscriptions:          mercury-users-request at csse.unimelb.edu.au
--------------------------------------------------------------------------