[m-users.] C foreign language interface question

[Julien Fischer]

Hi Fabrice,

On Wed, 21 Apr 2021, fabrice nicol wrote:

> I'm currently working on an interface library with R for Mercury, using the C 
> FFI of both languages as a bridge.
>
> This work in progress implies working with arrays, both in Mercury and the C 
> FFI, and preliminary results are encouraging.
>
> There is a couple of features, which I'm not sure are documented in the 
> current reference manual, that I would like to make clear before proceeding 
> further:
>
> 1.  The 'sharing' FFI foreign_proc annotation
>
> In chapter 15.1., this annotation is not documented, perhaps because it is 
> not advised to use it. Yet it looks like important in array.m to define 
> Mercury arrays, as shown in the excerpt below from array.m:1500, foreign_proc 
> init_2:
>
>      sharing(yes(int, T, array(T)), [
>                 cel(Item, []) - cel(Array, [T])
>
> Is this annotation necessary to allow Mercury to access C arrays? What does 
> it perform? I checked in compiler/parse_tree_out_pragma.m, where this 
> annotation is defined, but there is not enough there to make it safe use.

Those attributes are used by the experimental compile-time garbage
collection system; they are not required to allow Mercury to access C
arrays -- you should ignore them and you do not need to add them to
your own code.

Aside: the details of that system can be found in Nancy Mazur's Phd
thesis, which is available from our papers page.

     Compile-time garbage collection for the declarative language Mercury.
     Nancy Mazur. Ph.D. thesis, Catholic University of Leuven, Belgium, May 2004.

> 2. R has its own GC, which is fired by any call to the C FFI of R. To protect 
> recently allocated memory chunks from the R GC, the dedicated C FFI has a 
> useful macro named PROTECT, and another one to unprotect the said chunks, 
> leaving them open to R garbage collection. A quick-and-dirty reference is 
> here: https://github.com/hadley/r-internals/blob/master/gc-rc.md
>
> The question is simple. It is either (a) or (b):
>
> (a) if C memory chunks allocated by the R C FFI **are not** protected from 
> the Mercury GC, while the R-C FFI code is running:

What is the R C FFI using to allocate these chunks of memory?  Its own
GC, malloc() or is it using the Mercury GC?

> -->  is there a similar Mercury-C FFI macro, or a (possibly not 
> documented) Mercury FLI directive, or any other way, to make it so that these 
> chunks of C memory are no GC'd in competition with the R GC?
>
> (b) if C memory chunks allocated by the R-C FFI **are** protected from  the 
> Mercury GC when the R-C FFI code is running:
>
> -->  when the R server has been shut down, how to free these chunks 
> initially allocated by the R-C FFI code:
>
> - by calling the Mercury GC?
>
> I.E. by calling 'MR_GC_register_finalizer' as  in file cairo.m of the 
> mercury_cairo 'extras' library, whose design I overall follow. Is this the 
> right way?

The situation in mercury-cairo is that there are foreign resources
(foreign to Mercury that is) that have been created / allocated in some
way external to Mercury.  These foreign resources are being stored
inside Mercury allocated values and we want them to be destroyed /
deallocated when the containing Mercury value is GC'd.
Registering a finalizer on the Mercury allocated value is used to make
that happen.

> If I understand it well, 'MR_GC_register_finalizer' may call a specific C 
> memory cleanup procedure for non (Mercury) GC-managed memory chunks coming 
> from foreign interfaces?

There's some documentation for the Mercury memory allocation functions
in runtime/mercury_memory.h.  The function passed to
MR_GC_regsister_finzlier is user-defined, so in principle you can do
whatever you like in it.

> - or by bluntly calling 'free' in due time with a dedicated Mercury-C FFI 
> call, independently of any Mercury GC intervention ?

I'm not sure what you mean by that.

> Perhaps a few words on such issues would be useful in the FLI sections of the 
> Reference manual.
> Incidentally, perhaps this manual could document 
> MR_ArrayPtr in 15.3.1 (used in array.m). It may be of interest when Mercury 
> arrays are passed along to the C FFI (as in ML_resize_array for example).

I'll see what we can do about that.

Julien.