nested modules proposal

[Peter Schachte]

On Tue, 17 Feb 1998, Fergus Henderson wrote:

> Hmm, to my eyes it doesn't look quite as nice when you use `.' as the
> operator.
> 
>  	X = .a.b.c,
>  	Y = .a.b.d.
> 
> Another possibility, by the way, is to have some distinguished module
> name (e.g. `mercury', or perhaps `root') for the root module,
> and to not allow other modules to have the same name.
> Then you can write `mercury.std.io' or `mercury.my_module'
> and it is guaranteed that the name is a fully-qualified name.

.a.b.c looks nicer to me than mercury.a.b.c.  I find something strange about
the reserved module name, and something even stranger about it being
`mercury'.

> > Hmmm.  It seems quite odd to nest a module within another module's
> > *interface* section.  I assume the reason to do that would be to export
> > the inner module's exports from the outer module.
> 
> No, it is to export the inner modules *interface* from the outer module.
> (The exports are the things contained within the interface, but here
> we are exporting the interface itself.)

I don't understand the distinction.  What does it mean to export a module's
interface as opposed to the things it exports?  Is this the distinction
between exporting its name and its contents?

> > I think it would be much
> > nicer to handle it the other way:  some kind of declaration in the
> > implementation section of the outer module indicating that the inner
> > module's exports are exported from the outer module.  E.g.,
> > 
> > 	:- module foo.
> > 	:- interface.
> > 	:- export_module foo:bar.
> > 	...
> > 	:- implementation.
> > 	:- module foo:bar.
> > 	....
> 
> That would be bad, because it would mean that changing something in the
> implementation section of `foo' can affect foo's interface.

Depending on how you look at it...  Only changes in an interface section can
affect foo's interface, it's just that changes in bar's interface can as
well.

Anyway, I don't think that's always a bad thing.  When you're defining a
module, you may want to list all the things that should be in the interface,
but you may instead want to think of it as the sum of what's in some other
modules, in effect delegating the interface to the child modules.  I've
done this sort of thing without support for it, and found it quite a
nuisance to maintain the explicit list of exports.

If you don't want your module's interface to depend on the interfaces of
submodules, just list the exports explicitly.

> > In fact, the
> > nesting approach doesn't handle in any clean way I can think of what will
> > commonly be wanted:  each submodule has an interface to sibling submodules,
> > and a subset of that interface is made visible outside the parent module.
> 
> One way of achieving what you want with the current proposal is by
> having separate modules for the sibling interface and for the public
> interface (e.g. `foo.private_bar' and `foo.bar'), with the public
> interface constructed from the sibling interface by re-exporting the
> bits of that need to be public.  Yes, the re-exporting is ugly.
> 
> We could provide more direct support for this by adding a
> `:- sibling_interface' declaration, analagous to `:- interface'
> or `:- implementation' except that the declarations in the
> `:- sibling_interface' section are only accessible to sibling modules
> (or sub-modules of sibling modules).

This is beginning to sound like "friends" in c++.

> So I guess you could also argue for `:- cousin_interface',
> `second_cousin_interface', `third_cousin_interface', etc.

And don't forget uncle_interface, fourth_cousin_twice_removed_interface and
so on :-b.

This seems the wrong mental model to me.  If I'm writing a module, I
determine its interface to the outside world.  But I don't have any business
deciding which part of my interface is to become part of some other module's
interface.  Another module is perfectly entitled to export some, all, or
none of what it imports from my module (and in fact I can't stop her from
doing any of those).

I suppose it wouldn't hurt to add a "friend" feature to allow a particular
module access to a particular pred/function in this module, but I'm not
convinced it's necessary.  I think it's probably good enough to construct a
composite module from the few modules that need more intimate access to one
another, and have that module export all but the more intimate parts of the
submodules. 

> 	section			exported to
> 	-------			-----------
> 	interface		anyone
> 	sibling_interface	siblings
> 	implementation		sub-modules (i.e. children)
> 
> In fact, it might be better to change things so that the implementation
> is not visible in child modules, and instead add a `:- child_interface'
> for the part of the parent module which is visible in child modules.
> Then it would be
> 
> 	section			exported to
> 	-------			-----------
> 	interface		anyone
> 	sibling_interface	siblings
> 	child_interface		sub-modules (i.e. children)
> 	implementation		no-one

Oh, please, not more sections!

Here's another idea that simplifies things a bit more:  modules don't export
their contents to submodules at all.  It's easy enough to move the parts 
of the module that need to be seen by submodules into a new submodule,
which can then be imported as needed.

> Another alternative would be to use the Eiffel style of
> explicitly naming the modules to which you want to export
> each part of the interface; for example

This is more like the friends idea.  I'd argue to leave it out for now until
more than one person asks for it.

> 	:- module foo.
> 	:- interface.
> 	... public interface stuff ...
> 
> 	:- import_module siblings_only.
> 	:- module siblings_only.
> 	... stuff for siblings only ...
> 	:- end_module siblings_only.
> 
> 	:- implementation.
> 	...
> 
> Now, anyone who imports module `foo' gets just the public interface.
> If they want the sibling interface too, then they need to also
> explicitly use/import `foo.siblings_only' (because import_module
> only imports the top-level module, not any sub-modules).

But that leaves control in the wrong place:  with the importing module,
rather than the defining module.

> > This raises another point:  one would like to be able to construct a module
> > from several submodules, but the the outside world shouldn't see how the
> > outer module is constructed.  E.g., I've got predicates m:m1:p1 and m:m2:p2,
> > but I'd like them to be visible to users of module m as m:p1 and m:p2.  This
> > is important to allow the designer of module m to change the implementation
> > later.  Sure, this can be done by writing a slew of pass-through predicates,
> > but this is pretty awful.
> 
> Well, there's lots of things that might be nice.
> If we support all of them, then we may break the camel's back.

Always a danger, but I'd argue that this is really half of what nested
modules are needed for.  I'd argue it's the more important half.

Your perspective tends to be as a system implementor, for which what you're
proposing would probably be ok.  But look at it from the perspective of a
package developer.  Say you're developing an extensive GUI library.  Surely
you will want to implement this in several modules.  You will probably want
to present it to the user as a single module, because to present it as a
hierarchy of modules gives the user more information than she wants, and
also constrains your impementation and maintenance unduly.

On the other hand, for system building, the current Mercury module system
works ok.  Nested modules would be nice in a few places, but is not as
critical as the ability to (without *too* much cruft) compose a module from
several others. 

> If you allow a declaration such as
> 
> 	:- module m.
> 	:- interface.
> 	:- export_pred m1.p1.
> 
> to be equivalent to
> 
> 	:- module m.
> 	:- interface.
> 	:- pred p1(arg::mode) is det.
> 	:- implementation.
> 	p1(Arg) :- m1.p1(Arg).
> 
> then we have the unfortunate consequence that changing the interface
> to `m1' may implicitly change the interface to `m2'.
>
> Implicit interface changes are arguably a bad idea.  The interface
> should be a contract, and you don't want contracts that change without
> notice.  So arguably you should have to explicitly write in the
> pred declaration in the interface.

I don't think that's a good argument.  I'd agree you should be able to
explicitly write the pred declaration in the interface, but I don't think
you should have to. 

It's all in how you look at these composite modules.  If you think of them
as a sort of indirection, then it seems reasonable to refer the user to
other modules.  The interface extraction tool (I am still hacking on this 
occasionally, BTW) can be made to refer to the source files of submodules to
compose the interface of the parent module.

Think of it in a group project context.  One way to handle this would be to
have a single person who must approve all changes to the product's
interface.  This is similar to the model you're proposing.  Another
perfectly reasonable way to handle this would be for each developer to be in
charge of her part of the product's interface, in a decentralized process. 
The latter model is probably easier to work with. 

> Of course, it would be nice if there was a more convenient way of
> writing `p1(Arg) :- m1.p1(Arg).' particularly if there are lots of
> arguments.  It might be nice to allow just `p1 = m1.' as an
> alternative.

This would be good, too, for people who like the centralize model.  How
about:

	:- pred p1(...argtypes...) = submod:p2.
	:- mode p1(....) is ....
	....

> > In fact, I would argue that users of module m
> > should never be able to refer to m:m1:p1 or m1:p1 at all.
> 
> I don't agree with this.  I want to be able to refer to `std.list.append'
> and `std.string.append'.

Ok, I'll buy that.

> > > 	Every module gets an implicit `:- import_module std' declaration,
> > > 	unless it contains an explicit `:- use_module std' declaration.
> > 
> > So then is there no way to be sure you're not using the standard library
> > (because you're using an alternative standard library instead)? 
> 
> That's right.  (Well, there's always `grep'.)
> Got a better suggestion?

Just the obvious one:  some sort of

	:- no_standard_library.

declaration.  I guess on balance, it's not worth it.  But I don't really
like the idea that `:- use_module std.' actually takes away information.

> > Is the syntax for the module declaration at the top of the file for module
> > foo:bar:
> > 
> > 	:- module foo:bar.
> > or
> > 	:- module bar.
> > or
> > 	:- module :foo:bar.
> 
> All of these should be allowed.
> 
> My current implementation only supports the first and second one.
> It requires that you name the source file `foo.bar.m'.

How about this, then:

    1)	require the declaration:

		:- module :foo:bar

	(but maybe allow :- module foo:bar as a syntactic convenience).  So
	there's no need for an --in-package option.

    2)	As I said above, submodules don't see their parent module's
	contents, so there's no need to find the parent module to compile
	this module.  The module declaration gives you fully qualified names
	for everything.

    3)	Allow the module file to be named bar.m or foo.bar.m (try the latter
	first when looking for .int etc. files.)

    4)	A module A can see any module B whose module path up to its
	module name is an initial subpath of A.  So

		:a:b:c	can import	:d
		:a:b:c	can import	:a:d
		:a:b:c	can import	:a:b:d
		:a:b:c	cannot import	:d:e
		:a:b:c	cannot import	:a:d:e
		::b:c	cannot import	:a:b:d:e
		:a:b:c	already imports	:a:b:c:d

	(and similarly for `using' modules).

    5)	Have a `:- reexport_module module1, module2, ....' declaration to
	re-export everything exported from module1, module2, ....

    6)	Have a `:- include_module module1, module2, ....' declaration
	which, when it appears in the interface section, makes module1,
	module2, ... visible to modules that use/import this module.

    7)	Bob's your uncle.


-Peter Schachte			| A city is a large community where people are
pets at cs.mu.OZ.AU		| lonesome together -- Herbert Prochnow 
http://www.cs.mu.oz.au/~pets/	| 
PGP key available on request	|