[m-dev.] Minimal streams diff

[Ralph Becket]

Michael Day, Thursday,  3 October 2002:
> 
> > My thought was to reproduce much of the io module functionality for
> > streams, but to type specialize the stream-based operations for
> > std{in,out,err}.  This way we get good stream-based performance for
> > ordinary IO.
> 
> I can parse that sentence but can't quite understand its meaning. Could
> you perhaps show me a quick example involving write_char and stdout? Are
> you referring to the optimisation issue that Fergus brought up?

Yes, this is in response to Fergus' mention of the optimization problem.

The idea is that we extend the stream library with predicates like
read_line_as_string and write_int and format and so forth to use stream
based IO.

Then we add types to the io module that correspond to std{in,out,err}
and make them stream instances.  If the predicates in the stream module
are type specialized for these io types then, since there's only one
level of indirection between the modules, the stream based operations on
the io stream instances should not incurr a penalty (i.e. all the method
lookups etc. should get optimized away without the need for intermodule
optimization.)

> For those kind of applications, I wouldn't be using the stream interface,
> I would be using the underlying implementation directly (for example
> calling posix__write myself, rather than passing characters one by one to
> a stream and waiting for it to call posix__write). Perhaps indicates a
> need for a bulk stream interface that character streams can be layered
> over?

Could be.  My guess is that the common case is to send a bunch of bytes
for output, rather than just one at a time.

> When using posix__write directly, although the operating system will
> buffer it (not create 4096 IP packets each containing one byte...) I am
> trying to reduce the number of syscalls. Plus of course, if there is no
> underlying target platform, it will need to be done in Mercury...

My understanding was that these things were implemented so that the data
was buffered first in "user space" by the posix library before being
sent to the OS (which probably copies the data into another buffer...),
thereby incurring only one syscall per user space buffer flush.  If this
is so, then buffering yet again in Mercury is a mistake.

If the target platform does not do any buffering (wha???) then we could
supply an appropriate stream implementation for those cases.

> The buffering question seems to be more an issue of how the stream is
> created, and what its actually writing to/reading from, rather than an
> issue for the interface of the stream itself. Presumably once you've
> created a stream with a particular buffer size, it's rare that you want to
> change that buffer size dynamically?

True.

> As for non-blocking streams, that will require an additional interface,
> but that's something that can be added once blocking streams (the simpler
> case) have been dealt with.

That sounds fair.

- Ralph
--------------------------------------------------------------------------
mercury-developers mailing list
Post messages to:       mercury-developers at cs.mu.oz.au
Administrative Queries: owner-mercury-developers at cs.mu.oz.au
Subscriptions:          mercury-developers-request at cs.mu.oz.au
--------------------------------------------------------------------------