[m-users.] On package managers for Mercury

[Julian Fondren]

Hello list,

As noted, there's some prior prior work on a "something like CPAN
or mpm or gems" for Mercury. I looked around for a bit and found:

* Prior work

** Mercury Anarchy Archive
Last update: 2010-08
URL: http://manarchive.sourceforge.net/
ANN: 
http://lists.mercurylang.org/archives/users/2007-February/004353.html
CLI tool: manarchive-config
Package count: 6

I thought there was no content at all here until started to write
this very email, due to the SourceForge interface: you've gotta to
click on 'Code', not 'Files'.

The "Accessing Manarchive" instructions also no longer work, due to
SourceForge changes.

This is a shared filedump with a convenience tool and some
expectations about the content. With so much free code hosting
around, and with code being small enough that paid hosting will be
very cheap, all I see is downsides from the 'anarchic' part of
people being able to change my code without talking to me.

** Mercury Package Manager
Last update: 2015-08
URL: https://github.com/sebgod/mpm
ANN: (none?)
CLI tool: mpm
Package count: 0

This is slightly bit-rotten as it refers to io.poly_type, but once
that's fixed it compiles.

Typing 'make' is enough for it to install its executable into
$HOME/bin, which is a shocking deviation from civilized behavior.

I only dug into it enough to see that "listing available packages"
seems to be unimplemented behavior.

** Mercury Project Manager
Last update: 2019-05 (August: not actually bad for Mercury?)
URL: https://bitbucket.org/charles_shuller/mpm/src/master/
ANN: http://lists.mercurylang.org/archives/users/2019-March/008470.html
CLI tool: mpm
Package count: 0

This is a build system with a JSON project definition and a
dependency fetcher. So it's very appropriately named. You can't
discover someone else's Mercury package with it, though.

For build systems there's already mmc, mmake, and mmc --make, as
well as (simple, easy) makefiles and (alien, sanity destroying)
autoconf files.

I don't like JSON either. The point's made that XML is worse, but
what about Mercury's own term reading? Just imagine all the
validation code you don't have to write.

** Putting stuff on github and hoping people can find it through
    the hundreds of mis-identified Matlab and machine learning(?)
    files
URL: https://github.com/search?q=language%3AMercury&type=Repositories
ANN: if you post links to this list, people might find them
      eventually
CLI tool: none
Package count: 20 or so (hidden among 300+ hits)

This actually has a lot of advantages vs. the other work here. You
get free hosting without losing control (in practice).
Collaboration is easy and convenient. You don't need to use any
additional tools or change your workflow.

As a downside, there's not much you can expect for consistency, but
lots of inconsistent packages would be a nice problem to have, vs.
the current circumstance of very few packages.

The major downside is that if you'd just deleted your package
instead of uploading it to Github, it'd only be a little less
discoverable. Discoverability is so bad on Github that projects
like this exist, instead:

https://github.com/ohenley/awesome-ada (announced on January of
this year btw. This wouldn't be a bad thing to duplicate.)

And more:
https://github.com/sindresorhus/awesome#programming-languages

** Adding stuff to the 'extras' directory
Packages: 34

Of all the "prior work" I think this is actually the best one that
you can use right now. People that watch the Mercury github will
see commits to it, and see when packages are added. It's easy to
fork the Mercury repo and put your new extras in a branch that you
can then submit back to the repo in a PR. People can see from the
git log and from "main author" notices where the code came from.

I think this option just needs some encouragement, like a "How to
submit your package to extras/" mini-doc on
https://mercurylang.org/documentation/documentation.html

* What's the point of this anyway?

Without pointing any fingers, I think it's easy to cargo-cult the
benefit of a package manager. And then, perhaps it's easy to get
discouraged when the planes don't come even though you've built a
perfectly good something-or-other. If the manarchive has existed
for 12 years and there aren't even 12 packages on it, there must
not be a point to making a package manager for Mercury.

So I'll just answer the question. The point of a package manager is
social alchemy: it is to turn the /lead interaction/ into the /gold
interaction/.

** The lead interaction

- Alice discovers Mercury. "wow, what a cool language!"
- Alice wants to do some pedestrian, normal thing with it. "I can't
   start exim on my VPS for some nonsense reason to do with
   libperl.so. Why don't I just write an SMTP server in Mercury
   that's just good enough for my needs?"
- Alice writes net.m, a TCP socket library that's good enough for
   smtp.m
- Alice writes smtp.m, an SMTP library suitable for a mail service
   that can accept a low volume of email and then ask dovecot to
   deliver it locally.
- Alice writes tiny_recvmail.m
- Alice runs the server and eventually runs out of problems with it
- Alice throws the mature code on github and forgets about it

(time passes)

- Bob has a bunch of servers that run exim and, coincidentally,
   has tons of spam.
- Bob has the crazy idea of instrumentalizing exim to make it
   easy to study and react to spam.
   (this means writing lots of code in Perl.)
- Bob discovers Mercury. "wow, I'd much rather work with spam in
   this language!"
- Bob looks for any prior smtp work in Mercury, and doesn't find
   any
- Bob can't justify starting from the very beginning, before he's
   even sure that Mercury will help him.
- Bob moves on to other things

(more time passes)

- Alice moves on to other things
- the end

** The gold interaction

- Alice ... throws the mature code on github and adds it to a
   package manger

(time passes)

- Bob ... looks for any prior smtp work in Mercury, and finds
   tiny_recvmail.m
- "this is great!", but for Bob's purposes it's also very lacking.
   Bob creates a github issue listing what's lacking, and continues
   to work with the server.

- Alice gets an email about the issue and checks it out. "you want
   extensive logging? My VPS doesn't have a lot of space but if it's
   optional..."
- Alice looks for any prior work in logging in Mercury, and finds
   some
- Alice adds logging to her project even though she has no personal
   use for it

- Bob meanwhile has added some analysis hooks, essentially
   reinventing SpamAssassin as a module within the mail daemon.
- Bob submits this as an addition to tiny_recvmail

(this goes on for a while)

- Alice occasionally get paid contracts to work on mega_mungemail.m
- Bob solves the problem of spam, forever, albeit through the
   roundabout method of accidentally creating a murderous AI that
   decides that humanity is the problem
- the end

* So what does the alchemy really need, to work?

Just one thing: discoverability. That "X looks for prior work and
doesn't/does find it" is where the lead/gold outcome is decided.

Although calling that "just one thing" is like saying that all you
need to keep the rain off of you is a roof. Roofs don't just float
in the air above your head. So there are some good things to have
if you want people to actually discover anything:

1. a server on the internet that holds the list of packages

2. a website that you can use easily add a package to the list

3. a cli tool, preferably one included in Mercury, that can search
    through this list and grab packages from it

And then, there's obviously a lot more you can do. Packages could
have versions and versioned dependencies, you could have a safe
build system that can't let a malicious package hack your servers,
you could have standardization of package contents and continuous
integration and test failure reports. But the necessary minimal
core of a solution is a webserver, a CGI script, and a client tool
that can make some web requests and maybe call "git clone" for you.


So... I've got #1 and #2 done, and am working on #3 now. That'll
probably be released in a week or two. I'm sending this out now as
it's already quite a long email.


Cheers,
Julian