nested modules proposal

[Fergus Henderson]

On 18-Feb-1998, Peter Schachte <pets at students.cs.mu.OZ.AU> wrote:
> > > Hmmm.  It seems quite odd to nest a module within another module's
> > > *interface* section.  I assume the reason to do that would be to export
> > > the inner module's exports from the outer module.
> > 
> > No, it is to export the inner modules *interface* from the outer module.
> > (The exports are the things contained within the interface, but here
> > we are exporting the interface itself.)
> 
> I don't understand the distinction.  What does it mean to export a module's
> interface as opposed to the things it exports?  Is this the distinction
> between exporting its name and its contents?

Yep.

> This seems the wrong mental model to me.  If I'm writing a module, I
> determine its interface to the outside world.  But I don't have any business
> deciding which part of my interface is to become part of some other module's
> interface.

Sure, of course.

> I suppose it wouldn't hurt to add a "friend" feature to allow a particular
> module access to a particular pred/function in this module, but I'm not
> convinced it's necessary.

Right.  That's basically what I think too.  I think that if a particular
module needs more than the usual access to something defined in another
module, then idiom of using nested modules for "views" is a good enough
solution to this problem.

> I think it's probably good enough to construct a
> composite module from the few modules that need more intimate access to one
> another, and have that module export all but the more intimate parts of the
> submodules. 

Often that will work, but it is not a good solution in the general case.
Suppose A and B need to be "friends", and B and C need to be friends,
and C and D need to be friends.  If the only solution was to put them
all in a single package, then in the worst case I end up with everything
having to be in the one package.

> Here's another idea that simplifies things a bit more:  modules don't export
> their contents to submodules at all.  It's easy enough to move the parts 
> of the module that need to be seen by submodules into a new submodule,
> which can then be imported as needed.

That does simplify things a bit, but on the other hand it makes things
a bit non-orthogonal.  Currently, anything defined inside a module
-- types, insts, modes, preds, typeclasses, etc. --
has access to the other stuff defined in that module.
It would be a bit strange if modules didn't have that same access.

This suggestion would also make it impossible for a package to have
private sub-modules.

I'd call this the "Clayton's" nested modules proposal --
"The nested modules you have when you not having nested modules." ;-)

> > 	:- module foo.
> > 	:- interface.
> > 	... public interface stuff ...
> > 
> > 	:- import_module siblings_only.
> > 	:- module siblings_only.
> > 	... stuff for siblings only ...
> > 	:- end_module siblings_only.
> > 
> > 	:- implementation.
> > 	...
> > 
> > Now, anyone who imports module `foo' gets just the public interface.
> > If they want the sibling interface too, then they need to also
> > explicitly use/import `foo.siblings_only' (because import_module
> > only imports the top-level module, not any sub-modules).
> 
> But that leaves control in the wrong place:  with the importing module,
> rather than the defining module.

You're right that it is not perfect.
Nevertheless, I think that in practice it would be good enough.

> Your perspective tends to be as a system implementor, for which what you're
> proposing would probably be ok.  But look at it from the perspective of a
> package developer.  Say you're developing an extensive GUI library.  Surely
> you will want to implement this in several modules.  You will probably want
> to present it to the user as a single module, because to present it as a
> hierarchy of modules gives the user more information than she wants, and
> also constrains your impementation and maintenance unduly.

OK, fair enough.
You can do that, you just need to write some forwarding code.
If your interface is small, then the forwarding code won't be large.
If your interface is large, then you should probably split it up
into sub-modules anyway, so in that case this model doesn't really apply.

[Actually, this sort of thing has come up in the Mercury standard
library.  `term_io__read_term' is an example of this -- it's
implementation is in two separate modules (parser and lexer), and
term_io__read_term is just a forwarding predicate.]

> Think of it in a group project context.  One way to handle this would be to
> have a single person who must approve all changes to the product's
> interface.  This is similar to the model you're proposing.  Another
> perfectly reasonable way to handle this would be for each developer to be in
> charge of her part of the product's interface, in a decentralized process. 
> The latter model is probably easier to work with. 

Well, nested modules supports the latter model just fine.
You just have different modules gui.windows, gui.icons, gui.menus, etc.

> > Of course, it would be nice if there was a more convenient way of
> > writing `p1(Arg) :- m1.p1(Arg).' particularly if there are lots of
> > arguments.  It might be nice to allow just `p1 = m1.' as an
> > alternative.
> 
> This would be good, too, for people who like the centralize model.  How
> about:
> 
> 	:- pred p1(...argtypes...) = submod:p2.
> 	:- mode p1(....) is ....
> 	....

That's a poor syntax, because you're mixing interface and
implementation details in the one declaration.

I think just writing `p1 = submod.p2.' in the implementation
section would be a better syntax.  Or to avoid ambiguity
problems, `p1/3 = submod.p1/3.'

> > > > 	Every module gets an implicit `:- import_module std' declaration,
> > > > 	unless it contains an explicit `:- use_module std' declaration.
> > > 
> > > So then is there no way to be sure you're not using the standard library
> > > (because you're using an alternative standard library instead)? 
> > 
> > That's right.  (Well, there's always `grep'.)
> > Got a better suggestion?
> 
> Just the obvious one:  some sort of
> 
> 	:- no_standard_library.
> 
> declaration.  I guess on balance, it's not worth it.  But I don't really
> like the idea that `:- use_module std.' actually takes away information.

Yes, I guess it is a minor wart.
But as you say, `:- no_standard_library' does not really seem worth it.

> > > Is the syntax for the module declaration at the top of the file for module
> > > foo:bar:
> > > 
> > > 	:- module foo:bar.
> > > or
> > > 	:- module bar.
> > > or
> > > 	:- module :foo:bar.
> > 
> > All of these should be allowed.
> > 
> > My current implementation only supports the first and second one.
> > It requires that you name the source file `foo.bar.m'.
> 
> How about this, then:
> 
>     1)	require the declaration:
> 
> 		:- module :foo:bar
> 
> 	(but maybe allow :- module foo:bar as a syntactic convenience).  So
> 	there's no need for an --in-package option.

I'd rather not do that if it isn't necessary, for two reasons.
And it isn't always necessary; in particular it isn't necessary
if the file name has enough information to specify the
fully-qualified package name.

One reason why I'd rather not require it is just consistency:
fully-qualified names are not required when defining any other
entities, so why require them when defining sub-modules?  Would the
fully-qualified name be required only for non-nested sub-modules? 
The decision about whether a sub-module is defined in the same source
file or a different one is fairly arbitrary, so it would surely be
nicer if whether or not a sub-module is nested or not does not affect
the syntax required to define that sub-module.  On the other hand, if
you make the fully-qualified name a requirement even for nested
sub-modules, then you now have an inconsistency between modules and
other entities.

The other reason is that often you write a bunch of modules
and then later decide to package them.  It would be good
if packing them or changing their location in the package
hierarchy didn't require editing the source code of the
modules.

>     2)	As I said above, submodules don't see their parent module's
> 	contents, so there's no need to find the parent module to compile
> 	this module.  The module declaration gives you fully qualified names
> 	for everything.

This is the Java approach.

As I said above ;-), I don't really like this idea.
If we adopted this, then in what sense would submodules
really be components of the parent?
If we go for this, then all we have is a hierarchical namespace
for modules; we don't really have nested modules, and we don't
have any means for encapsulation at a higher level than a module.

I think that eventually we would like (physically) nested modules,
and for them, having proper scoping (i.e. having access to the
stuff in the parent module from within the nested module) is
pretty important.  If we're going to add something to support
packages (encapsulation units larger than a single source file),
it would be a major bonus it if we can later use the same mechanism
for encapsating units smaller than a single source file.
This is why I think it is worthwhile giving the nested modules
access to the implementation of the parent module.

Something like the Java approach is what I wanted when I first
started thinking about such things, just to keep it simple.
But having thought about it a bit, I think it's really not
that difficult for an implementation to support proper nesting.

I have however uncovered one potential practical problem:
if you change the parent module (e.g. by adding a new `:- include_module'
declaration), then all the sub-modules will need to be recompiled.
This would be a bit of a pain.

>     3)	Allow the module file to be named bar.m or foo.bar.m (try the latter
> 	first when looking for .int etc. files.)

OK...

>     4)	A module A can see any module B whose module path up to its
> 	module name is an initial subpath of A.  So
> 
> 		:a:b:c	can import	:d
> 		:a:b:c	can import	:a:d
> 		:a:b:c	can import	:a:b:d
> 		:a:b:c	cannot import	:d:e
> 		:a:b:c	cannot import	:a:d:e
> 		::b:c	cannot import	:a:b:d:e
> 		:a:b:c	already imports	:a:b:c:d
> 
> 	(and similarly for `using' modules).

You lost me here.
Does this mean `:user:foo' cannot import `:std:list'?

>     5)	Have a `:- reexport_module module1, module2, ....' declaration to
> 	re-export everything exported from module1, module2, ....

So this exports the _contents_ of the modules, not the modules
themselves, right?  If m1 defines foo, and m2 contains
`:- reexport_module m1', then do I refer to foo as `m2.foo',
not `m2.m1.foo'?

>     6)	Have a `:- include_module module1, module2, ....' declaration
> 	which, when it appears in the interface section, makes module1,
> 	module2, ... visible to modules that use/import this module.

Ah.  So this is an exception to rule 4?

I see.  Rules 4 & 6 give you a way of enforcing some encapsulation
for packages, by making some modules private to a package.

Another way of stating them would be that a package does not
have to list its sub-modules explicitly; it only needs to list
the public sub-modules.

So the differences between this proposal and my nested modules
proposal are

	2)		No nested scoping.
	5)		The `reexport_module' declaration.
	4&6)		No need to explicitly list the names of
			private sub-modules in the parent module.
	1) and 3)	Miscellaneous minor issues.

I think these are all mostly orthogonal issues which can be debated seperately.

With regard to 2), I've already outlined why I prefer nested scoping
(orthogonality, better support for physically nested modules,
and because it's really not that hard to implement).

With regard to 5), I think that typically you will not
want to reexport whole modules, but only parts of them
(otherwise why not just use `include_module' instead of `reexport_module'?).
I think better support for renaming (e.g. `p/3 = m1:p/3.', discussed above)
would be a better way of achieving this.

With regard to 4&6), would it be better to not require explicit
naming of the public sub-modules?  Public sub-modules seem to be
a lot more common.  (This would avoid the problem of having to recompile
everything in a package when you add a new module to the package.)

In summary, I'd say your proposal, excluding 5), is slightly easier
to implement, while my proposal is a bit more orthogonal and provides
more functionality.

-- 
Fergus Henderson <fjh at cs.mu.oz.au>   |  "I have always known that the pursuit
WWW: <http://www.cs.mu.oz.au/~fjh>   |  of excellence is a lethal habit"
PGP: finger fjh at 128.250.37.3         |     -- the last words of T. S. Garp.