[m-dev.] 0.13 release: op/3 syntax

[Mark Brown]

On 02-Mar-2006, doug.auclair at logicaltypes.com <doug.auclair at logicaltypes.com> wrote:
> Hi, back, Mark,
> 
> Thank you for your reasonable reply.  I concede the op/3 declaration for
> release 0.13.  But for none of the reasons you put forward.  I concede it
> because it's not something the Mercury team wants.  This is entirely
> perplexing to me:  this desire is directly contrary to your documentation:
> please explain, then, lines 250-253 in TODO of the compiler distribution

$ cvs annotate compiler/notes/todo.html

...
1.1          (aet      03-Apr-97): <li> implement user-defined operators: <br>
1.1          (aet      03-Apr-97):      Add a new construct `:- op(Pred, Type, Op).' as in Prolog;
1.1          (aet      03-Apr-97):      change prog_io.m to parse this construct and call io__op
1.1          (aet      03-Apr-97):      accordingly.  But how does this fit in with the module system?
...

Note both the revision number and the date.

> and
> wishlist item #1 of prog_io.m, lines 48-50.

$ cvs annotate compiler/prog_io.m

...
1.15         (fjh      28-Dec-93): % 1.  implement importing/exporting operators with a particular fixity
1.15         (fjh      28-Dec-93): %     eg. :- import_op prefix(+). % only prefix +, not infix
1.15         (fjh      28-Dec-93): %     (not important, but should be there for reasons of symmetry.)
...

So I guess that, technically, "abandoned" is too strong a word.  Marginally.

> >I'm not sure exactly which existing code you are referring to, but it's
> >probably only there for historical reasons.  As you may be aware, Mercury
> >started life by being bootstrapped from Prolog, which explains to a large
> >extent why it uses Prolog syntax.  Any support for op/3 is quite likely
> >to be left over from that era.
> 
> Are the declarations 'import_op', 'use_op' and 'export_op' NU Prolog
> declarations? I have not seen them in other Prolog distributions (XSB, SWI,
> Quintus, Sicstus). See lines 1188-1200 of prog_io.m (and elsewhere).

I can't find any other occurrences, either in the code or docs.  I think
those lines should have been removed long ago.

> 
> >What I _can_ say is that the idea of providing op/3 in Mercury was
> >abandoned a long time ago.  There was a majority view (not unanimous,
> >though) of people in the Mercury group that user defined operators almost
> >always make code harder to read for anyone other than the person who wrote
> >it.  This would particularly be the case with op/3's relationship with the
> >module system -- effectively each file has its own syntax, so in a large
> >project (e.g., hundreds or thousands of files) that's either a lot of
> >different syntaxes to understand, or else a lot of duplicated op/3
> >declarations.
> 
> Out customer is one of the proud owners of one of the world's largest, if
> not /the/ largest set of Prolog systems.  We have defined semantics for new op
> syntax in less than 10 modules (out of the many, many modules in our systems).
> In at least four critical areas the op syntax is an absolute necessity, for
> defining those systems any other way would put the regular syntax in the way
> of the problem solution.  We DO have the large project; we DO use the op/3
> syntax (very) sparingly and sensibly.

I realise that sufficiently disciplined programmers can use this feature
in a way that won't overburden people reading their code.  That's why I said
"almost always" instead of "always".  ;-)

The problem is that that doesn't give much guarantee or reassurance to
people who are going to have to read Mercury code.  I think we're convinced
of the usefulness of this feature to people writing Mercury code, but the
needs of people _reading_ Mercury code are at least as important as those
of people writing it (after all, there will generally be more people reading
the code than writing it!).

The fact that it is _possible_ to use this feature in a sensible way does
reduce the burden on readers to a minimal extent, but not near enough.  The
concerns I have for code consumers still far outweigh those that I have for
code producers.

> And, I am not alone in asserting that
> the op/3 declarations are necessary to /increasing/ the declarative nature
> of these systems.
> 
> >We decided long ago that backticks (e.g., `op`) had none of the disadvantages
> >of user defined operators, most of the advantages, and were almost as concise.
> 
> The `op` syntax is less useful that you put forward.  Requiring user-defined
> syntax be different than that of the language itself is a sure killer for
> that functionality

I'm not sure what you mean here.  Backticks are very much part of the
language.

> (cf. APL's user-defined functions in "Growing a Language"
> gls).  The Mercury team, itself, acknowledges this.  Example: the
> reimplementation of module pprint from `<>` to ++, and the conversion of the
> `..` operator to part of the regular language in release 0.11.

That's a different issue.  We of course recognise that operator syntax is
useful -- that's why we have it -- the issue here is whether the operator
table is a fixed part of the language or whether it can be extended on a
file-by-file basis, with no guarantees that different files will be
consistent in their extensions.

> See also the
> work-in-progress document:
> 
> http://www.logicaltypes.com/literature/mutable-syntax-in-mercury.html,
> 
> particularly section 1. of "Alternatives" and end-note 1 for further discussion
> of `op` syntax.

Thanks, that's a useful read.

> In fact, does anyone at all use this `op` syntax?

Yes.

> If so, is
> it because op/3 declarations are not available?

No.  It was because the operators in question were not considered general
enough to make them an official part of the language.

> I'll go one step further:
> `op` syntax should be stricken from the language.
> 
> Moreover, as I understand it, `op` syntax is binary.  70% of the operators we
> have defined are binary, yes, but the other operators are either prefix or
> postfix. `op` syntax only addresses part of the problem.

Yes, I agree with that.  The other limitation of backticks is that you only
get one precedence level, so parentheses are often required.  But this is a
Good Thing for people who need to read Mercury code -- figuring out how
an operator expression is parsed is hard enough already even with a fixed
operator table.  With an operator table that differs from file to file,
it would be much harder (unless the writer of the code was _very_
disciplined).

For example, how is the following term parsed:

:- some [T] impure pred foo(S::in, T::out) is det => baz(T) <= bar(S).

?

Where do I need to put the brackets to make this a valid Mercury declaration?

> 
> >The global extent was only part of the problem.  The main difficulty is
> >the one I mention above, and the one Ralph alludes to below: to understand
> >someone's code you first have to parse it in your mind, and to do that you
> >need to know exactly which user-defined operators apply to the file you are
> >looking at.
> 
> Granted.  In my experience, however, this is not difficult.  It is always
> immediately clear from me from the usage what the user-defined syntax (of
> course) /and/ semantics are.  Perhaps you have had different experiences.

Did you try putting correct brackets in the above declaration?  How many
tries did it take?

It took me several.  :-(

> 
> >I was going to ask about this [Mercury ops taking precedence over user-defined
> >ops], because the documentation you posted earlier
> >didn't make that clear (unless I missed something).  From briefly looking
> >at your code, it seems that it won't report a meaningful error message
> >if this rule is violated, or if you try to redefine an operator you have
> >previously defined.  This definitely needs to be addressed: 
> 
> I agree completely with you.
> 
> >the compiler throwing an exception is always considered a compiler bug.
> 
> I don't get this.  If the program the compiler is attempting to compile is
> incorrect, then the compiler quits, telling the user why.  Redefining
> already-defined Mercury syntax, if we agree this is a Bag Thing(tm) (and I
> think we do), /should/ stop the compilation process, as this is an exceptional
> case, just as calling :- foo(int::in) is det. with foo("bar") should stop the
> compilation process. Implementing code to do this in the compiler for user-
> defined operators is trivial.

I mean that rather than throw an exception, the compiler should report an
error in plain English in the (reasonably) standard format, including the
source file and line number.

> 
> >Personally, I'd hate to have to understand such code.  Having two predicates
> >with the same name but different semantics is bad enough.  Having two
> >operators with the same name but different precedence is truly diabolical,
> >IMHO.
> 
> So, you're calling the person who provides the implementation to do just that
> "Evil"?  Well, as long as you append "PhD" to it ("I didn't watch four minutes
> of a guy who went to four years of Evil medical school for nothing, thank you
> very much!"), I'm fine with my diabolical nature. :)

Hehe.  "Dr. Evil" sounds quite cool.  :-)

> >Another reason why we have never proceeded with user defined operators
> >is because, to my knowledge, no one has addressed the interactions with
> >the module system adequately.
> 
> "Adequately" is such a slippery word.  My implementation of op/3 is inadequate
> vis-a-vis modules for the Mercury team, but my colleagues and I find it
> entirely adequate for the range of applications we need to use it for and
> inadequate for none of them.  So, there it is.

When other people want a justification for why the language is designed the
way it was, they will expect it from us rather than you ;-).  We have to try
to consider what future users might need or want (and usually we have to do
this without them being around to provide input).

Hence, "adequately" is nonetheless appropriate.

> >An alternative approach to dealing with this issue is to implement a
> >preprocessor that exists entirely separately of the compiler.  It would
> >parse the module items exactly as your modified version of the parser does,
> >and then spit out items with all user defined operators in non-operator
> >form (and make liberal use of `:- pragma source_file' and `#line'
> >directives so that error messages are reported correctly).
> 
> This is absolutely the worst way to go about things.  See: my reference
> above, "Alternatives", sections 3 and 4.  Furthermore, does the Mercury team
> have a fully and completely defined grammar (EBNF, perhaps) of the Mercury
> language that successfully parses syntax as defined by the current compiler
> and language definition?  If so, are you willing to share that and keep that
> update as the syntax changes every release and between releases?

You don't need to go to that much trouble.  All you need to do is read in
one term at a time with `read_term_with_op_table', look for terms of the
form `':-'(op(_, _, _))' and update your op table if you find one, then
write the terms out in plain, non-operator syntax.  For best results, use
`:- pragma source_file' declarations whenever the file changes and `#line'
directives whenever the line changes.

> 
> >This would only need to be updated when Mercury's syntax changes, which is
> >relatively rare.
> 
> I think it is less rare than you think:  ROTD-2006-02-25 changed the syntax
> (see library/ops.m).  Release 0.12 changed the syntax from release 0.11 (see
> release 0.12 "Highlights").  Release 0.13 is adding (the very good and nice)
> functional dependency syntax.  Release 0.11 added state variable, and on, and
> on.

I mean relatively rare compared to changes on the development branch, which
you would have to merge in if you maintain a modified implementation of the
compiler.

> [acceptance criteria]
> >	- No language features get added to the repository until their
> >	  desirability is broadly accepted, the semantics is completely
> >	  and precisely defined, and their implementation is well tested
> >	  and passes peer review.
> 
> K.  So, only need to pass peer review. (broadly accepted: do you wish me to
> have two other companies forward you request for op/3?

I mean broadly accepted by the people who have the obligation to justify
the language design.  Namely, the Mercury developers.

For a full review, you should post the entire change in the format produced
by `cvs diff' at the top level.  It should include all the changes to
documentation as well.

(On this point, I would recommend that for significant language changes
the documentation should be modified first and reviewed prior to the
implementation.  That can save you a lot of trouble.)

> semantics defined: done.

Well, from the description you posted it wasn't clear to me whether duplicate
definitions would be accepted.  I had to look at the source code.

> Implementation well-tested: done;

Regression tests need to be provided for the Mercury test suite, covering
both legal and illegal uses of the feature.

> in production code: got it).
> 
> >	- Significant language features must be added early in the release
> >	  cycle.  Not late in the release cycle, and definitely not just
> >	  before release.  Such changes can only be validated effectively
> >	  if they have been put to use for a reasonable period of time, and
> >	  it is important that this validation has happened prior to the
> >	  feature being made available in a numbered release. (*)
> 
> Done.  I posted my changes, which have only changed to accomodate changes
> to the ROTD distributions, at the very beginning of release 0.12.2, "long"
> before preparations for release 0.13 were announced:
> 
> http://www.cs.mu.oz.au/research/mercury/mailing-lists/mercury-developers/mercury-developers.200601/0016.html

For "added" read "passed review and committed", not just posted for review.

Cheers,
Mark.

--------------------------------------------------------------------------
mercury-developers mailing list
Post messages to:       mercury-developers at cs.mu.oz.au
Administrative Queries: owner-mercury-developers at cs.mu.oz.au
Subscriptions:          mercury-developers-request at cs.mu.oz.au
--------------------------------------------------------------------------