[m-dev.] 0.13 release: op/3 syntax

[doug.auclair at logicaltypes.com]

Hi, back, Mark,

Thank you for your reasonable reply.  I concede the op/3 declaration for
release 0.13.  But for none of the reasons you put forward.  I concede it
because it's not something the Mercury team wants.  This is entirely
perplexing to me:  this desire is directly contrary to your documentation:
please explain, then, lines 250-253 in TODO of the compiler distribution and
wishlist item #1 of prog_io.m, lines 48-50.

Anyway, moving on from release 0.13.  I believe the addition of op/3
declarations is still justified.  I also feel the reasons the Mercury team has
put forward not to include op/3 are not strongly founded.  So, I believe and
am hopeful that we can come to an understand that is beneficial to all
involved.  So, to it:

You wrote:
>They are quite firm.  But that doesn't release us from the obligation to
>justify them, of course.  And since you've put quite a lot of effort into
>this, I think we owe it to you.

Thank you.

>I'm not sure exactly which existing code you are referring to, but it's
>probably only there for historical reasons.  As you may be aware, Mercury
>started life by being bootstrapped from Prolog, which explains to a large
>extent why it uses Prolog syntax.  Any support for op/3 is quite likely
>to be left over from that era.

Are the declarations 'import_op', 'use_op' and 'export_op' NU Prolog
declarations? I have not seen them in other Prolog distributions (XSB, SWI,
Quintus, Sicstus). See lines 1188-1200 of prog_io.m (and elsewhere).

>What I _can_ say is that the idea of providing op/3 in Mercury was
>abandoned a long time ago.  There was a majority view (not unanimous,
>though) of people in the Mercury group that user defined operators almost
>always make code harder to read for anyone other than the person who wrote
>it.  This would particularly be the case with op/3's relationship with the
>module system -- effectively each file has its own syntax, so in a large
>project (e.g., hundreds or thousands of files) that's either a lot of
>different syntaxes to understand, or else a lot of duplicated op/3
>declarations.

Out customer is one of the proud owners of one of the world's largest, if
not /the/ largest set of Prolog systems.  We have defined semantics for new op
syntax in less than 10 modules (out of the many, many modules in our systems).
In at least four critical areas the op syntax is an absolute necessity, for
defining those systems any other way would put the regular syntax in the way
of the problem solution.  We DO have the large project; we DO use the op/3
syntax (very) sparingly and sensibly.  And, I am not alone in asserting that
the op/3 declarations are necessary to /increasing/ the declarative nature
of these systems.

>We decided long ago that backticks (e.g., `op`) had none of the disadvantages
>of user defined operators, most of the advantages, and were almost as concise.

The `op` syntax is less useful that you put forward.  Requiring user-defined
syntax be different than that of the language itself is a sure killer for
that functionality (cf. APL's user-defined functions in "Growing a Language"
gls).  The Mercury team, itself, acknowledges this.  Example: the
reimplementation of module pprint from `<>` to ++, and the conversion of the
`..` operator to part of the regular language in release 0.11.  See also the
work-in-progress document:

http://www.logicaltypes.com/literature/mutable-syntax-in-mercury.html,

particularly section 1. of "Alternatives" and end-note 1 for further discussion
of `op` syntax.  In fact, does anyone at all use this `op` syntax?  If so, is
it because op/3 declarations are not available?  I'll go one step further:
`op` syntax should be stricken from the language.

Moreover, as I understand it, `op` syntax is binary.  70% of the operators we
have defined are binary, yes, but the other operators are either prefix or
postfix. `op` syntax only addresses part of the problem.

>The global extent was only part of the problem.  The main difficulty is
>the one I mention above, and the one Ralph alludes to below: to understand
>someone's code you first have to parse it in your mind, and to do that you
>need to know exactly which user-defined operators apply to the file you are
>looking at.

Granted.  In my experience, however, this is not difficult.  It is always
immediately clear from me from the usage what the user-defined syntax (of
course) /and/ semantics are.  Perhaps you have had different experiences.

>I was going to ask about this [Mercury ops taking precedence over user-defined
>ops], because the documentation you posted earlier
>didn't make that clear (unless I missed something).  From briefly looking
>at your code, it seems that it won't report a meaningful error message
>if this rule is violated, or if you try to redefine an operator you have
>previously defined.  This definitely needs to be addressed: 

I agree completely with you.

>the compiler throwing an exception is always considered a compiler bug.

I don't get this.  If the program the compiler is attempting to compile is
incorrect, then the compiler quits, telling the user why.  Redefining
already-defined Mercury syntax, if we agree this is a Bag Thing(tm) (and I
think we do), /should/ stop the compilation process, as this is an exceptional
case, just as calling :- foo(int::in) is det. with foo("bar") should stop the
compilation process. Implementing code to do this in the compiler for user-
defined operators is trivial.

>Personally, I'd hate to have to understand such code.  Having two predicates
>with the same name but different semantics is bad enough.  Having two
>operators with the same name but different precedence is truly diabolical,
>IMHO.

So, you're calling the person who provides the implementation to do just that
"Evil"?  Well, as long as you append "PhD" to it ("I didn't watch four minutes
of a guy who went to four years of Evil medical school for nothing, thank you
very much!"), I'm fine with my diabolical nature. :)

>Well, I think you've done a better job than Prolog.  But I'm not convinced
>by your assertion that the scope of an op/3 declaration should be from its
>position until the end of the file.  Assuming I am coerced into accepting
>the idea that the same operator may have different precedence in different
>files, I still think that others who use this feature would expect a more
>comprehensive interaction with the module system.  For example, that
>operators would be able to be exported (but when imported they should only
>affect the code in the module they are imported into, not the code imported
>from other modules as Prolog does).

Thank you. I put forward that operators /should not/ be exported.  This is
merely a preference, backed up by experience: I find that when a using module
puts the op declaration as well, it enhances the declarative nature of the
using module.  I'm also all for using modules being required to declare
which items they are using from which imported modules, a feature that
Mercury does not have but Quintus, Sicstus, and SWI permit (but do not
require).  It's so much easier for me to see in the source where the
semantics are coming from.  Tags probably do that for you, so there it is.

>Another reason why we have never proceeded with user defined operators
>is because, to my knowledge, no one has addressed the interactions with
>the module system adequately.

"Adequately" is such a slippery word.  My implementation of op/3 is inadequate
vis-a-vis modules for the Mercury team, but my colleagues and I find it
entirely adequate for the range of applications we need to use it for and
inadequate for none of them.  So, there it is.

[snip]
>What is compile-time for the code using op/3 is run-time for the parser
>that parses that code.  I think this is what is usually meant by a
>dynamic grammar, and what Ralph was referring to.

That still does not answer the question: are ops.m and read_term..._ops_table
deprecated?  And if dynamic syntax is a problem, how will the new yacc-like
parser handle `op` syntax?  The (useless to this discussion) material point
here is that if ops.m and read_term..._with_op_table go away, I've got some
work to do getting back op/3 into our local Mercury compiler, and I'd like to
know how future parsing will impact our stuff.

>An alternative approach to dealing with this issue is to implement a
>preprocessor that exists entirely separately of the compiler.  It would
>parse the module items exactly as your modified version of the parser does,
>and then spit out items with all user defined operators in non-operator
>form (and make liberal use of `:- pragma source_file' and `#line'
>directives so that error messages are reported correctly).

This is absolutely the worst way to go about things.  See: my reference
above, "Alternatives", sections 3 and 4.  Furthermore, does the Mercury team
have a fully and completely defined grammar (EBNF, perhaps) of the Mercury
language that successfully parses syntax as defined by the current compiler
and language definition?  If so, are you willing to share that and keep that
update as the syntax changes every release and between releases?

>This would only need to be updated when Mercury's syntax changes, which is
>relatively rare.

I think it is less rare than you think:  ROTD-2006-02-25 changed the syntax
(see library/ops.m).  Release 0.12 changed the syntax from release 0.11 (see
release 0.12 "Highlights").  Release 0.13 is adding (the very good and nice)
functional dependency syntax.  Release 0.11 added state variable, and on, and
on.  All these syntax changes are good, very good.  All these syntax changes
are significant.  This is the opposite of a gripe: a living language should
and does change.  Mercury is a living (viable) language.  It changes.

[acceptance criteria]
>	- No language features get added to the repository until their
>	  desirability is broadly accepted, the semantics is completely
>	  and precisely defined, and their implementation is well tested
>	  and passes peer review.

K.  So, only need to pass peer review. (broadly accepted: do you wish me to
have two other companies forward you request for op/3? semantics defined:
done.  Implementation well-tested: done; in production code: got it).

>	- Significant language features must be added early in the release
>	  cycle.  Not late in the release cycle, and definitely not just
>	  before release.  Such changes can only be validated effectively
>	  if they have been put to use for a reasonable period of time, and
>	  it is important that this validation has happened prior to the
>	  feature being made available in a numbered release. (*)

Done.  I posted my changes, which have only changed to accomodate changes
to the ROTD distributions, at the very beginning of release 0.12.2, "long"
before preparations for release 0.13 were announced:

http://www.cs.mu.oz.au/research/mercury/mailing-lists/mercury-developers/mercury-developers.200601/0016.html

>We can continue the debate on the best design for this new feature, but it
>doesn't meet our release criteria and won't be included in the 0.13 release.

Understood.  I hope you understand my position:  op/3 is good and necessary,
and I hope it to be included as part of the Mercury language.  The Mercury
team's position, up to this point, has a documented preference for including
op/3.  I hope the Mercury team can come back to this view, and I hope that my
implementation is useful for you when you do.

At any rate: op/3 out for release 0.13; no hard feelings; and let's come to
a good resolution for a future release quickly.

Sincerely,
Doug Auclair

--------------------------------------------------------------------------
mercury-developers mailing list
Post messages to:       mercury-developers at cs.mu.oz.au
Administrative Queries: owner-mercury-developers at cs.mu.oz.au
Subscriptions:          mercury-developers-request at cs.mu.oz.au
--------------------------------------------------------------------------