[m-dev.] thread.spawn_native

[Peter Wang]

On Wed, 11 Jun 2014 12:35:53 +1000, Paul Bone <paul at bone.id.au> wrote:
> On Wed, Jun 11, 2014 at 12:20:43PM +1000, Peter Wang wrote:
> > 
> > The problem is when two Mercury contexts try to use the same library
> > simultaneously.  They cannot rely on thread-local state to persist
> > across calls:
> > 
> >     1. context-1 calls liba.set(1) -- performed on IO-thread-1
> >     2. context-2 calls liba.set(2) -- also performed on IO-thread-1
> >     3. context-1 calls liba.get -- performed on IO-thread-1
> > 
> 
> Why doesn't each context use a handle or pointer to refer to it's own set of
> data in liba.  liba's API is very limited if it doesn't allow for this.

That's the problem.  If the API accepts context handles everywhere
instead of using thread-local state then we wouldn't be having this
discussion.

Here are a couple of examples:

libxml2: Although there are context handles, and it allows for parsing
multiple documents simultaneously in different OS threads, it crashes if
you try to parse multiple documents simultaneously (multiple Mercury
contexts multiplexed onto the same OS thread).  The context handle
probably does not encapsulate ALL the state.

OpenGL: you call glXMakeContextCurrent and from then on all GL calls
from that OS thread work within that rendering context.
Different threads may work with different contexts simultaneously.
GL functions do not take a rendering context argument.

Allegro 5: sorry... I blame OpenGL

I'm sure I could find more examples easily.  Yes, the APIs are to blame.
I guess the reason is that they start out with global state, then thread
support is bolted on using thread-local state for source compatibility.

Only libxml2 is a *actual* problem right now.  We can work around it if
we really care, but we'll just use hlc.par.gc.

Anyway, this is a secondary issue.  The primary issue I want to solve is
blocking functions.  Both spawn_native and IO workers can work.  I see
spawn_native as much simpler, and also happens to solve the API issue.
Your objection is that it may break assumptions in the implementation
currently made for efficiency.  I'm willing to try it and see.

On the other hand, I object to the complexity of IO workers and see no
benefit.  Some form of helper threads may be required to augment async
I/O in the future.  I don't think the two forms are necessarily the
same: spawn_native can be an escape hatch when something is not possible
in the helper thread, e.g. thread-local state, or blocking functions
that call back into Mercury.

Peter