[m-dev.] proposed additions: file copying in stdlib and compiler

[Zoltan Somogyi]

On 2023-11-07 16:32 +11:00 AEDT, "Julien Fischer" <jfischer at opturion.com> wrote:
> 
> 1. Add copy_file and copy_directory to the standard library.
> 
> Many programming languages provide some form of file copying operation as part
> of their standard libraries (Files.copy() in Java, File.Copy() in C#,
> std.filesystem::copy in C++17, std::fs::copy in Rust etc).  While file copying
> is in general is an intrinsically filesystem-specific thing, I think it is
> possible to provide a version that is sufficiently useful to warrant its
> inclusion standard library.

I agree: that should be possible.

> I propose we add the predicates io.file.copy_file and io.file.copy_directory to
> the standard library.  The actual semantics of these operations (e.g. how they
> deal with existing files, symbolic links etc) will need to be decided, but I'm
> intending to replicate what "cp" and "cp" -R do respectively.  (See the second
> half of this proposal below for why.)

I think we would want to agree on the semantics first. And for that, we would
need to know (a) what the possible set of implementation approaches are
for each backend/platform combo,  (b) what their semantics are, and (c) if
there is more than one feasible approach for a backend/platform combo,
their relative speed.

I am familiar with the Unix/C options, but not the others.

> For copy_file, I propose to shift the do_copy_file predicate from
> compiler/module_cmds.m into library/io.file.m and use that as the fall-back
> implementation.

Agreed.

> If the target platform provides a more efficient mechanism (e.g
> CopyFile() and friends from the Windows API), we would use that in preference.

Am I correct in presuming that you would get this info from autoconfigured options?

> Once we have copy_file, copy_directory can be implemented in Mercury using
> existing standard library predicates.

Agreed.

> 2. Copying files by the compiler
> 
> The Mercury compiler copies files and directories by invoking the command
> specified by the --install-command and --install-command-dir options via a call
> to system() or alike. (In practice, this means "cp" and "-R" respectively.)
> There are a couple of issues with this:
> 
> a. On some systems (e.g. Windows), this approach is really slow. While some of
>    the file copying in the compiler may be avoidable (viz. Zoltan's recent
>    proposal for interface files), some of it (e.g. installation targets) is not.
> 
> b. On Windows specifically, this approach creates a dependency on an external
>    cp-like program. While this is ok if you are using Mercury with MSYS
>    or Cygwin, it complicates packaging up Mercury directly for Windows.
>    (In previous packages for Windows, we used the cp from the UnxUtils ports
>    but they no longer seem to be maintained.)
> 
>    The various file copying utilities that are part of Windows (copy.exe,
>    xcopy.exe, robocopy.exe) aren't currently usable -- at a minimum we
>    would need to teach the compiler how to use them, they don't avoid
>    the call to system() and Microsoft seem incapable of creating a
>    tool that doesn't spew output all over the console.

I don't know enough about Windows to propose an acceptable alternative.
 
> I propose to add a new flag to the Mercury compiler, --install-method.
> This option will accept two values: external and internal.

I don't think those names are descriptive enough, but don't have
an alternate name to propose.
 
> With a value of "external", the compiler will use the values of
> --install-command and --install-command-dir to copy files and directories
> (just as it does now).

Obviously, that's fine.

> With a value of "internal", the compiler will use the copy_file and
> copy_directory predicates added to the standard library in the first part of
> this proposal to copy files and directories (or, if there is not agreement to
> add those predicates, from a new module in the compiler itself.)

The compiler will use ... for what purpose? For copying X.int.tmp files
to X.int files, I don't see a reason why we need to let users configure
what means the compiler should use; the autoconfigured method
should always be the best method available. For installs: that task
has traditionally been done *outside* the compiler, so compiler
options can't influence how that is done. Unless you are proposing
that the compiler should take over the job of installation?

Zoltan.