nested modules proposal

[Fergus Henderson]

On 25-Feb-1998, Peter Schachte <pets at students.cs.mu.OZ.AU> wrote:
> > > Here's another idea that simplifies things a bit more:  modules don't export
> > > their contents to submodules at all.  It's easy enough to move the parts 
> > > of the module that need to be seen by submodules into a new submodule,
> > > which can then be imported as needed.
> > 
> > That does simplify things a bit, but on the other hand it makes things
> > a bit non-orthogonal.  Currently, anything defined inside a module
> > -- types, insts, modes, preds, typeclasses, etc. --
> > has access to the other stuff defined in that module.
> >
> > It would be a bit strange if modules didn't have that same access.
> 
> And currently modules don't have access to anything they don't import/use.

Yes, this would be a change; but not a non-orthogonality.

> It would be a bit strange if modules suddenly had access to stuff without
> asking for it.

Well, a type declaration has access to stuff without the type declaration
itself asking for it, so it is consistent.

> It would mean that it would no longer be possible to read a
> module by itself (after referring to its imports), you'd have to read all
> ancestor modules as well.

True.  Ancestor modules are like imports in this respect,
except that you have to read both the interface and implementation
of the ancestors, whereas for imports you only have to read the interface.

So it would be good coding style to keep parent modules small
(just as it is good coding style is to keep interfaces small).

> Also note that with the model I'm proposing, it would make sense to include
> the same module in multiple other modules (I know your current code
> generation scheme can't handle this, but it a possibility for other
> implementations), whereas it would never make sense with your model.

Well, if allowing this would rule out common code generation schemes
such as the one we have, then I don't think it would be a good idea.

Actually it could still make sense to include the same source file
in multiple other modules, even with my model and our current code
generation scheme; it would be a different module each time,
and might even have different semantics each time,
so you'd have to compile to a different object file each time,
but with a more flexible source file <-> module name mapping
and with the `--in-module' option, it could work.

> > This suggestion would also make it impossible for a package to have
> > private sub-modules.
> 
> Not at all.  I explain how below.
> 
> > I'd call this the "Clayton's" nested modules proposal --
> > "The nested modules you have when you not having nested modules." ;-)
> 
> Who's Clayton?

Never mind.

> > > > Now, anyone who imports module `foo' gets just the public interface.
> > > > If they want the sibling interface too, then they need to also
> > > > explicitly use/import `foo.siblings_only' (because import_module
> > > > only imports the top-level module, not any sub-modules).
> > > 
> > > But that leaves control in the wrong place:  with the importing module,
> > > rather than the defining module.
> > 
> > You're right that it is not perfect.
> > Nevertheless, I think that in practice it would be good enough.
> 
> This is not very different than having multiple interface sections, and
> having the documentation extraction tool only show the first.  siblings_only
> is still part of the public interface for module foo.
> 
> I think, as I wrote above, the better solution would be to have a
> foo_internal module that exports both the public and sibling interface of
> foo, which is imported by siblings, and then have foo (re-)export only the
> public part of foo_internal.  This seems much more natural to me.  I really
> don't see anything to recommend your approach over mine.

Well, it doesn't require having a documentation extraction tool,
it avoids the need for convenient re-export mechanisms
it avoids the double-maintenance problem required by re-export
mechanisms (even the convenient ones would still have some
double-maintenance problems in this situation), and it avoids
the use of interface delegation (which is arguably bad style,
particularly if you don't have a documentation extraction tool).

As Tom said to me the other day, interface delegation makes things
easier for the person writing the interface, but harder for the
people using it.

Note that your `:- reexport_module' declaration would not be of direct
help here, since we want to reexport only part of foo_internal.
On the other hand, we could combine these two techniques, making
the public interface part of foo_internal a nested sub-module
`foo_internal.public', and the defining `foo' by reexporting
`foo_internal.public'.  Still, it's not the most elegant solution
imaginable.

> > > Your perspective tends to be as a system implementor, for which what you're
> > > proposing would probably be ok.  But look at it from the perspective of a
> > > package developer.  Say you're developing an extensive GUI library.  Surely
> > > you will want to implement this in several modules.  You will probably want
> > > to present it to the user as a single module, because to present it as a
> > > hierarchy of modules gives the user more information than she wants, and
> > > also constrains your impementation and maintenance unduly.
> > 
> > OK, fair enough.
> > You can do that, you just need to write some forwarding code.
> > If your interface is small, then the forwarding code won't be large.
> > If your interface is large, then you should probably split it up
> > into sub-modules anyway, so in that case this model doesn't really apply.
> 
> You may or may not want to split it up even if it's large.  I probably
> wouldn't want to.  And if it is split up, the way you choose to split it up
> for external consumption may not have much to do with how you want to split
> it up to implement it.  You may have similar concepts you want to present as
> separate modules but which is most convenient to implement as a single
> module.  You also may very well want to break up one user-visible module
> into several internal modules for implementation.  In these cases
> "re-bundling" is important, and should be made as simple for the programmer
> as possible. 

I guess we just disagree about how important and how simple.

> > > How about this, then:
> > > 
> > >     1)	require the declaration:
> > > 
> > > 		:- module :foo:bar
> > > 
> > > 	(but maybe allow :- module foo:bar as a syntactic convenience).  So
> > > 	there's no need for an --in-package option.
> > 
> > I'd rather not do that if it isn't necessary, for two reasons.
> > And it isn't always necessary; in particular it isn't necessary
> > if the file name has enough information to specify the
> > fully-qualified package name.
> 
> It doesn't seem like a good idea to me to rely on file naming conventions. 
> Especially one that will certainly not work under MS-DOG, VM/CMS, or even
> any unix with short file names.  You wouldn't want to make the `:- module'
> declaration in a file optional (since the file name gives you that
> information) would you? 

Yes, the compiler currently does that.
It is very convenient for experimentation.
I want to continue to support it.
(Even though it is a pain to implement.)

> If you go with the child-can't-see-the-parent's-contents model, then I don't
> think this is too important.  If the child can see the parent, then I think
> it is important that the source file contain enough information for the
> reader to find the parent, putting it in the Mmakefile or in the file name
> isn't good enough.

This is a reasonably good argument.

> > One reason why I'd rather not require it is just consistency:
> > fully-qualified names are not required when defining any other
> > entities, so why require them when defining sub-modules?
> 
> Because it's the module declaration that determines what the
> implicit qualification for other declarations is.

Ditto.

> >  Would the
> > fully-qualified name be required only for non-nested sub-modules?
> 
> Take your pick:  consistency or convenience.

Well, consistency with non-nested sub-modules and consistency
with other nested constructs (types, modes, etc.) are at odds,
so I'd pick convenience.

> > If we adopted this, then in what sense would submodules
> > really be components of the parent?
> 
> You'd tend to use submodules for packaging and interface engineering, which
> I think is the greater need.  Using them for mini-modules in the middle of a
> larger module would be somewhat less convenient, but I think this is a
> lesser need.  You could still do it easily when the mini-module doesn't need
> anything from the surrounding module, which is probably what I'd tend to
> want anyway (eg, for a type and its accessors).

Well, so long as the type doesn't include any fields whose types
are private to the parent module...

> > If we go for this, then all we have is a hierarchical namespace
> > for modules
> 
> Yes.
> 
> > ; we don't really have nested modules, and we don't
> > have any means for encapsulation at a higher level than a module.
> 
> I think I disagree.  What do you mean?

I meant you don't get private modules.
But I had misunderstood your proposal.

> > Something like the Java approach is what I wanted when I first
> > started thinking about such things, just to keep it simple.
> > But having thought about it a bit, I think it's really not
> > that difficult for an implementation to support proper nesting.
> 
> In that case, fine.  My suggestion was just to make it simpler to use and to
> implement.

OK.

> I'm still not comfortable, though, with putting whole a module
> in another module's interface section.  It just seems wrong to have a
> implementation of one module lexically within another's interface.

Yes, I agree with you on that one.
I'll change it.

> > >     4)	A module A can see any module B whose module path up to its
> > > 	module name is an initial subpath of A.  So
> > > 
> > > 		:a:b:c	can import	:d
> > > 		:a:b:c	can import	:a:d
> > > 		:a:b:c	can import	:a:b:d
> > > 		:a:b:c	cannot import	:d:e
> > > 		:a:b:c	cannot import	:a:d:e
> > > 		::b:c	cannot import	:a:b:d:e
> > > 		:a:b:c	already imports	:a:b:c:d
> > > 
> > > 	(and similarly for `using' modules).
> > 
> > You lost me here.
> > Does this mean `:user:foo' cannot import `:std:list'?
> 
> No, because std is imported everywhere.  It does mean that :user:foo can't
> import :belch:list without first using/importing :belch.

So am I right that

          4)	A module A can see any module B whose module path up to its
	        module name is an initial subpath of A.  So

should be

          4)	A module A can see any module B whose module path up to its
 	        module name is an initial subpath of A, or of any module
		imported or used by A.

?

> Yes.  I had assumed rules 4 and 6 were pretty much as in your proposal.  If
> not, what's the difference?

Nothing, I think.

> > With regard to 4&6), would it be better to not require explicit
> > naming of the public sub-modules?
> 
> This might be a very good solution to part of the problem, but it's more
> related to point 5.  If I have a predicate p in a module m, and m is
> "contained" somehow in module s, I'd like to be able to arrange any of the
> following
> 
> 	a)  users/importers of s can't see anything about m.
> 	b)  users/importers of s can see p as :s:m:p or m:p (importers only)
> 	    or p, if :s:m is also imported.
> 	c)  :s:p or p (importers only), but never :s:m:p or m:p.

a) make it a private sub-module (declare p in the implementation section of s)
b) make it a public sub-module (declare p in the interface section of s)
c) use delegation

> >  Public sub-modules seem to be
> > a lot more common.  (This would avoid the problem of having to recompile
> > everything in a package when you add a new module to the package.)
> 
> I don't understand what you mean by public submodules (b or c?). 

b.

> And I don't see why you have to recompile a module when you include a new
> submodule.

The sub-module implicitly imports the parent module,
so any change to the parent interface may change the legality
or semantics of the child module.

For example, if the child has `:- import_module parent.child2',
and the parent does not have `:- include_module child2', then
the child has an error.  Thus adding/removing an `:- include_module'
declaration can change the legality of the child.

-- 
Fergus Henderson <fjh at cs.mu.oz.au>   |  "I have always known that the pursuit
WWW: <http://www.cs.mu.oz.au/~fjh>   |  of excellence is a lethal habit"
PGP: finger fjh at 128.250.37.3         |     -- the last words of T. S. Garp.