nested modules proposal

[Peter Schachte]

On Wed, 18 Feb 1998, Fergus Henderson wrote:

> > I think it's probably good enough to construct a
> > composite module from the few modules that need more intimate access to one
> > another, and have that module export all but the more intimate parts of the
> > submodules. 
> 
> Often that will work, but it is not a good solution in the general case.
> Suppose A and B need to be "friends", and B and C need to be friends,
> and C and D need to be friends.  If the only solution was to put them
> all in a single package, then in the worst case I end up with everything
> having to be in the one package.

I think that's probably ok.  The point is that when you're implementing
submodule m of package p, you have in mind an interface to the other modules
that make up package p, as well as an interface for the users of package p.
So module m exports the union of these interfaces, and then package p
(re-)exports only the external part of m's interface.  Alternatively, you
could have a module m1 that imports m and (re-)exports the  external part of
m's interface.  Then p can either export the module m1, or just all of its
contents, depending on what you want p's interface to look like.

> > Here's another idea that simplifies things a bit more:  modules don't export
> > their contents to submodules at all.  It's easy enough to move the parts 
> > of the module that need to be seen by submodules into a new submodule,
> > which can then be imported as needed.
> 
> That does simplify things a bit, but on the other hand it makes things
> a bit non-orthogonal.  Currently, anything defined inside a module
> -- types, insts, modes, preds, typeclasses, etc. --
> has access to the other stuff defined in that module.
>
> It would be a bit strange if modules didn't have that same access.

And currently modules don't have access to anything they don't import/use.
It would be a bit strange if modules suddenly had access to stuff without
asking for it.  It would mean that it would no longer be possible to read a
module by itself (after referring to its imports), you'd have to read all
ancestor modules as well.

Also note that with the model I'm proposing, it would make sense to include
the same module in multiple other modules (I know your current code
generation scheme can't handle this, but it a possibility for other
implementations), whereas it would never make sense with your model.

> This suggestion would also make it impossible for a package to have
> private sub-modules.

Not at all.  I explain how below.

> I'd call this the "Clayton's" nested modules proposal --
> "The nested modules you have when you not having nested modules." ;-)

Who's Clayton?

> > > 	:- module foo.
> > > 	:- interface.
> > > 	... public interface stuff ...
> > > 
> > > 	:- import_module siblings_only.
> > > 	:- module siblings_only.
> > > 	... stuff for siblings only ...
> > > 	:- end_module siblings_only.
> > > 
> > > 	:- implementation.
> > > 	...
> > > 
> > > Now, anyone who imports module `foo' gets just the public interface.
> > > If they want the sibling interface too, then they need to also
> > > explicitly use/import `foo.siblings_only' (because import_module
> > > only imports the top-level module, not any sub-modules).
> > 
> > But that leaves control in the wrong place:  with the importing module,
> > rather than the defining module.
> 
> You're right that it is not perfect.
> Nevertheless, I think that in practice it would be good enough.

This is not very different than having multiple interface sections, and
having the documentation extraction tool only show the first.  siblings_only
is still part of the public interface for module foo.

I think, as I wrote above, the better solution would be to have a
foo_internal module that exports both the public and sibling interface of
foo, which is imported by siblings, and then have foo (re-)export only the
public part of foo_internal.  This seems much more natural to me.  I really
don't see anything to recommend your approach over mine.

> > Your perspective tends to be as a system implementor, for which what you're
> > proposing would probably be ok.  But look at it from the perspective of a
> > package developer.  Say you're developing an extensive GUI library.  Surely
> > you will want to implement this in several modules.  You will probably want
> > to present it to the user as a single module, because to present it as a
> > hierarchy of modules gives the user more information than she wants, and
> > also constrains your impementation and maintenance unduly.
> 
> OK, fair enough.
> You can do that, you just need to write some forwarding code.
> If your interface is small, then the forwarding code won't be large.
> If your interface is large, then you should probably split it up
> into sub-modules anyway, so in that case this model doesn't really apply.

You may or may not want to split it up even if it's large.  I probably
wouldn't want to.  And if it is split up, the way you choose to split it up
for external consumption may not have much to do with how you want to split
it up to implement it.  You may have similar concepts you want to present as
separate modules but which is most convenient to implement as a single
module.  You also may very well want to break up one user-visible module
into several internal modules for implementation.  In these cases
"re-bundling" is important, and should be made as simple for the programmer
as possible. 

> > How about this, then:
> > 
> >     1)	require the declaration:
> > 
> > 		:- module :foo:bar
> > 
> > 	(but maybe allow :- module foo:bar as a syntactic convenience).  So
> > 	there's no need for an --in-package option.
> 
> I'd rather not do that if it isn't necessary, for two reasons.
> And it isn't always necessary; in particular it isn't necessary
> if the file name has enough information to specify the
> fully-qualified package name.

It doesn't seem like a good idea to me to rely on file naming conventions. 
Especially one that will certainly not work under MS-DOG, VM/CMS, or even
any unix with short file names.  You wouldn't want to make the `:- module'
declaration in a file optional (since the file name gives you that
information) would you? 

If you go with the child-can't-see-the-parent's-contents model, then I don't
think this is too important.  If the child can see the parent, then I think
it is important that the source file contain enough information for the
reader to find the parent, putting it in the Mmakefile or in the file name
isn't good enough.

> One reason why I'd rather not require it is just consistency:
> fully-qualified names are not required when defining any other
> entities, so why require them when defining sub-modules?

Because it's the module declaration that determines what the
implicit qualification for other declarations is.

>  Would the
> fully-qualified name be required only for non-nested sub-modules?

Take your pick:  consistency or convenience.

> The other reason is that often you write a bunch of modules
> and then later decide to package them.  It would be good
> if packing them or changing their location in the package
> hierarchy didn't require editing the source code of the
> modules.

It would be good, but it's a pretty minor inconvenience for a pretty major
operation.

> >     2)	As I said above, submodules don't see their parent module's
> > 	contents, so there's no need to find the parent module to compile
> > 	this module.  The module declaration gives you fully qualified names
> > 	for everything.
> 
> This is the Java approach.
> 
> As I said above ;-), I don't really like this idea.
> If we adopted this, then in what sense would submodules
> really be components of the parent?

You'd tend to use submodules for packaging and interface engineering, which
I think is the greater need.  Using them for mini-modules in the middle of a
larger module would be somewhat less convenient, but I think this is a
lesser need.  You could still do it easily when the mini-module doesn't need
anything from the surrounding module, which is probably what I'd tend to
want anyway (eg, for a type and its accessors).

> If we go for this, then all we have is a hierarchical namespace
> for modules

Yes.

> ; we don't really have nested modules, and we don't
> have any means for encapsulation at a higher level than a module.

I think I disagree.  What do you mean?

> Something like the Java approach is what I wanted when I first
> started thinking about such things, just to keep it simple.
> But having thought about it a bit, I think it's really not
> that difficult for an implementation to support proper nesting.

In that case, fine.  My suggestion was just to make it simpler to use and to
implement.  I'm still not comfortable, though, with putting whole a module
in another module's interface section.  It just seems wrong to have a
implementation of one module lexically within another's interface.  But then
it always seemed weird to me to import module A within B's interface,
and not have that mean that B (re-)exports the public contents of A.

> >     4)	A module A can see any module B whose module path up to its
> > 	module name is an initial subpath of A.  So
> > 
> > 		:a:b:c	can import	:d
> > 		:a:b:c	can import	:a:d
> > 		:a:b:c	can import	:a:b:d
> > 		:a:b:c	cannot import	:d:e
> > 		:a:b:c	cannot import	:a:d:e
> > 		::b:c	cannot import	:a:b:d:e
> > 		:a:b:c	already imports	:a:b:c:d
> > 
> > 	(and similarly for `using' modules).
> 
> You lost me here.
> Does this mean `:user:foo' cannot import `:std:list'?

No, because std is imported everywhere.  It does mean that :user:foo can't
import :belch:list without first using/importing :belch.

> >     5)	Have a `:- reexport_module module1, module2, ....' declaration to
> > 	re-export everything exported from module1, module2, ....
> 
> So this exports the _contents_ of the modules, not the modules
> themselves, right?  If m1 defines foo, and m2 contains
> `:- reexport_module m1', then do I refer to foo as `m2.foo',
> not `m2.m1.foo'?

Yes.

> >     6)	Have a `:- include_module module1, module2, ....' declaration
> > 	which, when it appears in the interface section, makes module1,
> > 	module2, ... visible to modules that use/import this module.
> 
> Ah.  So this is an exception to rule 4?

Kind of.  Rule 4 says that you can't see into a module you haven't
used/imported (and further, you can only refer to :a:b as just b if you've
imported :a, if you've just used a, then you have to call it :a:b or a:b).
Rule 6 says you can't refer to :a:b at all unless :a makes b public.

> I see.  Rules 4 & 6 give you a way of enforcing some encapsulation
> for packages, by making some modules private to a package.

Yes.  I had assumed rules 4 and 6 were pretty much as in your proposal.  If
not, what's the difference?

> Another way of stating them would be that a package does not
> have to list its sub-modules explicitly; it only needs to list
> the public sub-modules.

No, I hadn't intended that.  You still have to have a `:- include_module'
declaration for all submodules.

> So the differences between this proposal and my nested modules
> proposal are
> 
> 	2)		No nested scoping.

I think this is a misleading way to put it, but I know what you mean.

> 	5)		The `reexport_module' declaration.
> 	4&6)		No need to explicitly list the names of
> 			private sub-modules in the parent module.

No.

> 	1) and 3)	Miscellaneous minor issues.
> 
> With regard to 5), I think that typically you will not
> want to reexport whole modules, but only parts of them
> (otherwise why not just use `include_module' instead of `reexport_module'?).

Because you want to create modules by merging other modules.  It's a matter
of interface engineering:  you build new interfaces out of old ones by
composition.  Sometimes you will want to re-export only part of a module,
and I think there should be a concise syntax for this (where you just list
which parts are to be reexported, without repeating the declarations).  And
yes, I realize this will make a module's interface depend on the submodule's
interface.  But since the module hierarchy must be a tree (or at least a
dag), this shouldn't be a problem.

> I think better support for renaming (e.g. `p/3 = m1:p/3.', discussed above)
> would be a better way of achieving this.

I like this, but I think this isn't enough, as it doesn't give a way for a
module to delegate (part of) its interface to another module.

> With regard to 4&6), would it be better to not require explicit
> naming of the public sub-modules?

This might be a very good solution to part of the problem, but it's more
related to point 5.  If I have a predicate p in a module m, and m is
"contained" somehow in module s, I'd like to be able to arrange any of the
following

	a)  users/importers of s can't see anything about m.
	b)  users/importers of s can see p as :s:m:p or m:p (importers only)
	    or p, if :s:m is also imported.
	c)  :s:p or p (importers only), but never :s:m:p or m:p.

But I'd also like to be to arrange any of these things independently of what
other submodules of s can see of m (well not independently, only a subset of
what siblings can see should be visible outside).

>  Public sub-modules seem to be
> a lot more common.  (This would avoid the problem of having to recompile
> everything in a package when you add a new module to the package.)

I don't understand what you mean by public submodules (b or c?).  And I
don't see why you have to recompile a module when you include a new
submodule.

> In summary, I'd say your proposal, excluding 5), is slightly easier
> to implement, while my proposal is a bit more orthogonal and provides
> more functionality.

I don't agree about orthogonality, but otherwise this is true.  But
*including* 5, I think my proposal provides more functionality, since it
allows interface delegation, and this allows the functionality of what you
call nesting without much fuss.


-Peter Schachte			| Do, or do not. There is no 'try'. -- Yoda
pets at cs.mu.OZ.AU		| ('The Empire Strikes Back') 
http://www.cs.mu.oz.au/~pets/	| 
PGP key available on request	|