[m-dev.] thread.spawn_native

[Paul Bone]

On Tue, Jun 10, 2014 at 04:34:47PM +1000, Peter Wang wrote:
> 
> The proposal is unclear to me.
> 
> Let's say the annotation is called `may_not_migrate'.  If a Mercury
> context calls a `may_not_migrate' foreign proc, then it is permanently
> tied to some Mercury engine (OS thread) from then on?  Presumably the
> foreign proc is executed directly in that OS thread, and not rerouted to
> an IO thread as in the case of blocking foreign procs.  Otherwise, it
> would not solve the API issue.
> 
> Where does the Mercury engine come from?  If the number of Mercury
> engines is fixed, doesn't it mean that a blocking `may_not_migrate'
> foreign proc will prevent progress of other Mercury threads?  Or can
> `may_not_migrate' foreign procs not block?
> 
> Can two Mercury contexts be bound to the same Mercury engine?
> Presumably so, if the number of Mercury engines is fixed.
> But then one context could stomp on the thread-local state expected
> by another context.

Good point.  Previously I hadn't thought about how these two features would
interact.  Because there's no clear way to find out aheas of time if any of
the foreign calls will block or not, then any may_not_migrate calls must be
done on an IO Worker thread (A pthread that is not part of the mormal set of
Mercury Engines).

Yes, my intention is that multiple contexts could be bound to the same
Mercury engine.  I beleive the managment of a thread-local state that's used
by foreign code is the responsability of the foreign code.  Mercury cannot
protect some foreign code from stepping on some other foreign code's state.

> > 
> > What if your explicit concurrency thread executes a parallel conjunction? or
> > is executed by a parallel conjunction.  We at least have to accept the
> > possibility.
> 
> Right, any explicit concurrency thread (of which the main thread is one)
> should be able to make goals available for execution by the parallel
> execution workers.  And the problem is that a parallel conjunct could
> (indirectly) call a foreign proc which has a requirement to be executed
> on a particular Mercury engine (OS thread).

There is nothing special about either the engines that are used to execute
sparks and contexts created by parallel conjunctions, nor is there anything
special about the sparks or contexts themselves.  So "parallel execution
worker" doesn't really mean anything, it's just any Mercury engine.

> It seems solvable.  When the parallel worker hits a foreign proc (say,
> with the `may_not_migrate' annotation) it should be able to make the
> work available to be picked up by the original Mercury engine.  I think
> that would be as if the parallel conjunct never left the original
> Mercury engine?

I think that idea works.  There is already support for handing work to specific
engines, it's used when a foreign call calls some Mercury code, then returns
back through the C code, it arranges for that return to be done on the
engine that originally made the foreign call so that it can use the same C
stack frame.

However if all may_not_migrate calls are made on IO workers then no
migration is necessary.

> Blocking foreign procs should not be executed in a parallel worker
> either.

As above, this is true for any Mercury Engine.

> > > I also think it's not only a temporary solution (and there are degrees
> > > of "temporary").  Even if we eventually move to your preferred model,
> > > the API issue would still remain.
> > 
> > Sorry, I forgot to mention the second proposal above.
> > 
> > > > 
> > > > I also think that thread.join could be made mandatory per-thread.  For
> > > > example, Java has the concept of normal threads and deamon threads.  To
> > > > clean up a normal thread you must call join, but this is not necessary for a
> > > > deamon thread.  Also the process will only exit once there are zero normal
> > > > threads running, but any number of deamon threads may be active when the
> > > > process ends (but we don't need that behaviour (I think)).
> > > 
> > > Sounds pretty ugly.
> > 
> > The benifit is that it allows developers to choose what behaviour they need.
> 
> Admittedly everything else is going down so it's not as bad
> as a thread.kill procedure.
> 

I agree.


-- 
Paul Bone