nested modules proposal

[Fergus Henderson]

On 17-Feb-1998, Peter Schachte <pets at students.cs.mu.OZ.AU> wrote:
> Firstly, I think this looks much simpler and cleaner than the previous
> (package) approach. 
> 
> > 	A module may contain sub-modules, delimited by
> > 	`:- module' and `:- end_module declarations'.
> 
> I assume you mean "`:- end_module' declarations" (note the single quotes).
> 
> Can the :- end_module declaration still be omitted for submodules written in
> separate files?

Yes.

> How about for the last nested module in a file, eg:
> 
> 	:- module foo.
> 	...
> 	:- module bar.
> 	...
> 	<eof>

Uh, I don't suppose it matters too much.  Probably we ought to
require an explicit `end_module bar' declaration in cases like this.
It would certainly be good style to include one.
But if a check to enfornce this doesn't happen to fall out of the
implementation all by itself, then adding one would be a very low
priority.

> > 	This implies that module qualifiers may contain a list of
> > 	module names (eventually using the syntax `m1.m2.foo', but for
> > 	the moment using the syntax `m1:m2:foo' or `m1__m2__foo')
> > 	rather than just a single module name.
> > 	(We should also add some new syntax for fully-qualified
> > 	module names, e.g. `:m1:m2:foo', but I can't think
> > 	of a workable syntax for it.)
> 
> I don't see the problem.  Under Quintus:
> 
> 	| ?- current_op(X,Y,(:)).
> 
> 	X = 600,
> 	Y = xfy ;
> 
> 	no
> 	| ?- op(600,fy,(:)).
> 
> 	yes
> 	| ?- X = a:b:c.
> 
> 	X = a:b:c
> 
> 	| ?- X = :a:b:c.
> 
> 	X = :a:b:c 
> 
> The problem arises when you want a *postfix* and infix op of the same name. 

Oh yes, you're right.

Hmm, to my eyes it doesn't look quite as nice when you use `.' as the
operator.

 	X = .a.b.c,
 	Y = .a.b.d.

Still, I suppose I could get used to that.

Another possibility, by the way, is to have some distinguished module
name (e.g. `mercury', or perhaps `root') for the root module,
and to not allow other modules to have the same name.
Then you can write `mercury.std.io' or `mercury.my_module'
and it is guaranteed that the name is a fully-qualified name.
This is the approach that Ada uses (the root module in
Ada is called `Standard', which is a bad choice IMHO).

> > 	The interface of a module does not include the implementations
> > 	of any sub-modules defined in that module's interface.
> 
> Hmmm.  It seems quite odd to nest a module within another module's
> *interface* section.  I asssume the reason to do that would be to export
> the inner module's exports from the outer module.

No, it is to export the inner modules *interface* from the outer module.
(The exports are the things contained within the interface, but here
we are exporting the interface itself.)

> I think it would be much
> nicer to handle it the other way:  some kind of declaration in the
> implementation section of the outer module indicating that the inner
> module's exports are exported from the outer module.  E.g.,
> 
> 	:- module foo.
> 	:- interface.
> 	:- export_module foo:bar.
> 	...
> 	:- implementation.
> 	:- module foo:bar.
> 	....

That would be bad, because it would mean that changing something in the
implementation section of `foo' can affect foo's interface.

> In fact, the
> nesting approach doesn't handle in any clean way I can think of what will
> commonly be wanted:  each submodule has an interface to sibling submodules,
> and a subset of that interface is made visible outside the parent module.

When I first read your mail, I thought that you were right.
But I thought of a nice idiom for doing this.

Ada has a similar limitation; I haven't heard anyone on comp.lang.ada
complaining.  So does Java.  Actually Java is much more limited.

One way of achieving what you want with the current proposal is by
having separate modules for the sibling interface and for the public
interface (e.g. `foo.private_bar' and `foo.bar'), with the public
interface constructed from the sibling interface by re-exporting the
bits of that need to be public.  Yes, the re-exporting is ugly.

We could provide more direct support for this by adding a
`:- sibling_interface' declaration, analagous to `:- interface'
or `:- implementation' except that the declarations in the
`:- sibling_interface' section are only accessible to sibling modules
(or sub-modules of sibling modules).

	section			exported to
	-------			-----------
	interface		anyone
	sibling_interface	siblings
	implementation		sub-modules (i.e. children)

In fact, it might be better to change things so that the implementation
is not visible in child modules, and instead add a `:- child_interface'
for the part of the parent module which is visible in child modules.
Then it would be

	section			exported to
	-------			-----------
	interface		anyone
	sibling_interface	siblings
	child_interface		sub-modules (i.e. children)
	implementation		no-one

But in the same way that say `mercury.std.io' might want to export things
to other modules in `mercury.std', e.g. `mercury.std.pipes', without
exporting them globally, if the namespace is a bit more nested then
might want to do the same thing at the next level up; e.g.
`mercury.std.io' might want to export the same things to
`mercury.extras.sockets' but not globally.
So I guess you could also argue for `:- cousin_interface',
`second_cousin_interface', `third_cousin_interface', etc.

At this point the design space gets quite large ;-)

Another alternative would be to use the Eiffel style of
explicitly naming the modules to which you want to export
each part of the interface; for example

	:- module mercury.std.io.
	:- interface.
	...
	:- interface for mercury.std.pipes, mercury.extras.sockets.
	% only the modules named can access this stuff
	...
	:- implementation.
	...
	:- end_module.

However, all of the above extensions are quite complicated, and
there is a simple alternative idiom using just the facilities of
the current proposal which gives you something which is very
close to the Module-3 idea of supporting multiple named interfaces
("views") for a single module.  The idea is to use nested modules
in the interface.  For example:

	:- module foo.
	:- interface.
	... public interface stuff ...

	:- import_module siblings_only.
	:- module siblings_only.
	... stuff for siblings only ...
	:- end_module siblings_only.

	:- implementation.
	...

Now, anyone who imports module `foo' gets just the public interface.
If they want the sibling interface too, then they need to also
explicitly use/import `foo.siblings_only' (because import_module
only imports the top-level module, not any sub-modules).
Note that the `import_module siblings_only' declaration means
that the module foo itself can access the stuff in the sub-module
without any qualifiers.

The only thing missing is that the compiler does not ensure that
only siblings of `foo' import the `foo.siblings_only' sub-module,
But that can be a coding style issue.
This idiom lets you have arbitrarily complicated conventions about
who is allowed to use a particular interface sub-module.

> This raises another point:  one would like to be able to construct a module
> from several submodules, but the the outside world shouldn't see how the
> outer module is constructed.  E.g., I've got predicates m:m1:p1 and m:m2:p2,
> but I'd like them to be visible to users of module m as m:p1 and m:p2.  This
> is important to allow the designer of module m to change the implementation
> later.  Sure, this can be done by writing a slew of pass-through predicates,
> but this is pretty awful.

Well, there's lots of things that might be nice.
If we support all of them, then we may break the camel's back.
It's a question of finding the largest possible subset of idioms
which we can support as well as possible with the least complexity.

If you allow a declaration such as

	:- module m.
	:- interface.
	:- export_pred m1.p1.

to be equivalent to

	:- module m.
	:- interface.
	:- pred p1(arg::mode) is det.
	:- implementation.
	p1(Arg) :- m1.p1(Arg).

then we have the unfortunate consequence that changing the interface
to `m1' may implicitly change the interface to `m2'.
Implicit interface changes are arguably a bad idea.  The interface
should be a contract, and you don't want contracts that change without
notice.  So arguably you should have to explicitly write in the
pred declaration in the interface.

You may not agree with this argument, but it does seem questionable
to add language support for a feature whose use is itself of
questionable style.

Of course, it would be nice if there was a more convenient way of
writing `p1(Arg) :- m1.p1(Arg).' particularly if there are lots of
arguments.  It might be nice to allow just `p1 = m1.' as an
alternative.

> In fact, I would argue that users of module m
> should never be able to refer to m:m1:p1 or m1:p1 at all.  I would argue
> that you only want to be able to see the module hierarchy "above you" and at
> your level, never below (or one level below, depending on what you think of
> as your level).  So if there's a predicate :m1:m2:m3:p, and it's exported by
> all three enclosing modules and imported into the current context, I should
> be able to specify it as p or m1:p or :m1:p in the current context, but not
> as m2:m3:p or m1:m2:m3:p or anything else.

I don't agree with this.  I want to be able to refer to `std.list.append'
and `std.string.append'.  I don't what to have to refer only to
`std.append'.

Also, it should be possible for both std.int.'+' and std.float.'+' to
be visible in the same place, and for the compiler to use overload
resolution to figure out which one it is, rather than for there to be
only a single `std.+' predicate.

> > 	Every module gets an implicit `:- import_module std' declaration,
> > 	unless it contains an explicit `:- use_module std' declaration.
> 
> So then is there no way to be sure you're not using the standard library
> (because you're using an alternative standard library instead)? 

That's right.  (Well, there's always `grep'.)
Got a better suggestion?

> > 	The language semantics of an `include_module' declaration
> > 	is that it is exactly equivalent to replacing the
> > 	`import_module' declaration with the module to which it
> > 	refers.
> 
> Except that the normal Mercury source file restrictions apply: the included
> file must be a single module (perhaps with modules nested within it).

Yes, I forgot to explicitly mention that.

> Is the syntax for the module declaration at the top of the file for module
> foo:bar:
> 
> 	:- module foo:bar.
> or
> 	:- module bar.
> or
> 	:- module :foo:bar.

All of these should be allowed.

My current implementation only supports the first and second one.
It requires that you name the source file `foo.bar.m'.

To support the second one while also allowing a more
flexible naming scheme, either

	- `mmc' needs a `--in-package' option,
	   and you need to compile your file with `--in-package foo'
	   (this could be done implicitly if you are using Mmake)

and/or

	- if you name the souce file foo/bar.m,
	  then you need to compile it from the parent directory,
	  i.e. `mmc -C foo/bar.m'.

> > 	The mapping between module names and file names is
> > 	is implementation-dependent.
> 
> A fine idea, unless you're prepared to carefully consider all possible file
> systems.  For starters, foo.bar.m is going to be tough under MS-DOG, and
> separate directories won't work under VM/CMS.

That's exactly why it should be implementation-dependent!

> And I shudder to think what
> any of this means in my washing machine or toaster ;-).

Well gee, you shouldn't try to compile code on your washing machine!
Cross-compile *to* your washing machine, of course.

If you're talking about the mapping between module names and object code
file names (rather than source file names), then I guess it would make
sense to use URLs instead of file names, and have your washing machine
download the code from the net.

> > 	Mmake gets a new `--in-module' option.
> 
> This appears to conflict with your later comments about file/module naming,
> suggesting that the file/module name would determine containing module(s). 
> Why is this switch needed?

Only for allowing more flexible naming conventions.

> And how do you write the default mmake rule for
> building .o files from .m's? 

If we allow a more flexible naming convention,
then `mmake depend' could (somehow) figure out the
mapping between module names and file names,
and spit out hard-coded rules in the dependency files
rather than relying on pattern rules.

> What about allowing sibling modules to use or import one another?  I hope
> this is allowed; it's pretty important.

Yes, of course.  All declarations in the parent module are visible
in child modules, including `:- module' (or `:- include_module')
declarations.  That implies that sibling modules can import one another.

> Can a nested module m automatically
> see everything imported/used into the parent module p,

Yes.

> including what p imports/uses/includes from sibling modules?

For imports/uses, yes.  For includes, no.

> What does it mean to *import* a nested module?  Is this an error?

No, it is not an error.  It means that all the declarations in the
interface of that nested module are accessible, without needing to
use an explicit module qualifier.

For example, this can be useful if you are using the nested module purely
for encapsulation, rather than for namespace control, as in the idiom
for module "views" bove.

-- 
Fergus Henderson <fjh at cs.mu.oz.au>   |  "I have always known that the pursuit
WWW: <http://www.cs.mu.oz.au/~fjh>   |  of excellence is a lethal habit"
PGP: finger fjh at 128.250.37.3         |     -- the last words of T. S. Garp.