[m-dev.] Re: for review: extras/dynamic_linking

[Fergus Henderson]

On 07-Dec-1998, Bert Thompson <aet at cs.mu.OZ.AU> wrote:
> A further possibility is to provide an interface to the Win32 dynamic
> loading calls, and add a module that provides the same functionality
> but abstracts away from the platform-specific implementations. Worth
> seeing how Erlang does this?

I've heard that tcl has a widely-ported abstraction layer for dynamic
loading, so if someone wants to make it more portable, that might
be a good place to start from.

I considered doing this.  But it's a bit more work, and it makes things
a bit more complicated.  Also there is an implementation of dlopen()
etc. in terms of the Win32 dynamic loading calls available for Cygwin,
so it may not be necessary.

> You may've done this already, but it would be good to look at the
> interfaces of dlsym etc on important Unixes such as Solaris and HP-UX.
> They're pretty uniform these days, fortunately.

Yep.  I didn't check HP-UX, but I checked Debian Linux,
Solaris, Digital Unix, and Irix.  The only difference was
that some of them did not support `RTLD_GLOBAL'; the code has a
`#ifdef' for that.

> Yet another thing that would be nice is to store type-infos in some
> standard format in the shobject and do load-time interface checking.
> For instance, you could have a compiler switch --generate-type-info-
> table that creates a pred of known type called "type_table", or some
> such thing, in which the types of all the module's exports are stored.

I agree that such a feature would be nice, but doing that would be
difficult, and a simple name-equivalence type check would not be
sufficient to ensure run-time type safety anyway, since it wouldn't
solve the problem of mixing code compiled with different versions
of the same module.  One way to solve that problem would be to do
a deep structural equivalence test, but that would be even more difficult.
It could also be potentially very costly -- as well as checking
the argument types, and any types used in their definitions 
(recursively), you'd have to check the equivalence of all the type
classes, and any types or type classes that they use (recursively).
I'm not sure that the costs of keeping all that information around
and checking it at run-time would be worth the benefits.

An appropriate coding style can reduce the number and complexity
of the calls to dl__sym and hence the likehood of run-time type errors.
In particular, when we have exisential data types, rather than having
a module which exports an ADT and lots of access procedre for that ADT,
requiring you to call dl__sym once for each procedre in the dynamically
loaded module that you want to call, you can instead define the interface
for that module as a typeclass and export just a single procedure that
gives you an instance of the typeclass.
E.g. instead of defining the module as

	:- module foo.
	:- interface.
	:- pred p1(t1::in, t2::out) is det.
	:- pred p2(t3::in, t4::out) is semidet.
	...
	:- implementation.
	:- end_module foo.

and then having module bar dynamically load in module foo and then
call dlsym for each of p1, p2, etc., with the danger each time of
getting the argument types, modes or determinism wrong, you can instead
define an interface module

	:- module foo_interface.
	:- interface.
	:- typeclass foo(T) where [ ... ].
	:- type any_foo ---> some [T] mk_foo(T) <= foo(T).

and have the module foo itself just export a single function
that provides a way for people to create an instance of that interface:

	:- module foo_impl.
	:- interface.
	:- import_module foo_interface.

	:- func foo_impl(...) = any_foo.

	:- implementation.

	:- type foo_impl -> mk_foo_impl(...).
	:- instance foo(foo_impl) where [ ... ].

	foo_impl(...) = mk_foo(mk_foo_impl(...)).

Now you only have to call dlsym once, for the function `foo_impl'.

This uses existential data types; I'm not sure off-hand if
there may be a way to do it without that.

If you're going to be doing a lot of complex stuff with dynamic loading,
then using the CORBA interface may be a better way to go.

> Ensuring there are no dangling references when unloading would
> be nice, but it is very difficult to do correctly.

Actually, that might not be too hard, using GC finalizers.

> Maybe some of this stuff could be `possible extensions' comments in the code?

Well, there's lots of possible extensions.  It probably wouldn't
hurt to document them, but I don't think it is necessary to do so.
It's important to document limitations, and the documentation does do so.
E.g. it makes it clear that the stuff is implemented using dlopen(), etc.,
which is not completely portable, and it makes it clear that the
interface is not type-safe.  It doesn't document the behaviour of dl__close
except to say that it is an interface to dlclose(), but the documentation
of dlclose() makes it clear that dangling references result in undefined
behaviour.

-- 
Fergus Henderson <fjh at cs.mu.oz.au>  |  "Binaries may die
WWW: <http://www.cs.mu.oz.au/~fjh>  |   but source code lives forever"
PGP: finger fjh at 128.250.37.3        |     -- leaked Microsoft memo.