[m-users.] Function to remove unicode byte order mark

[Peter Wang]

On Sat, 25 Feb 2017 08:57:58 +0100, Dirk Ziegemeyer <dirk at ziegemeyer.de> wrote:
> Hi,
> 
> I’d like to share a litte piece of code that removes the byte order mark (BOM) from the top of an utf-8 file.
> 
> I didn’t even know that a byte order mark exists until I saved an Excel table as Unicode.txt in order to be able to read it with Mercury. BOM is preserved during conversion to utf-8.
> 
> As Mercury doesn’t remove the BOM by itself, it’s up to the application to deal with it.
> 
> Dirk
> 
> 
> 
>     % Remove optional unicode byte order mark (BOM) from the beginning
>     % of an utf-8 file
>     %
> :- func remove_byte_order_mark(string) = string.
> 
> remove_byte_order_mark(RawFirstLine) = FirstLine :-
>     ( if
>         string.first_char(RawFirstLine, FirstChar, Rest),
>         char.to_int(FirstChar) = 0xfeff % Unicode byte order mark (BOM)
>     then FirstLine = Rest
>     else FirstLine = RawFirstLine
>     ).

Hi Dirk,

A slight improvement is to use the (in, in, uo) mode of first_char to
avoid copying the rest of the string unless the first char is the BOM,

    first_char(S, '\xFEFF\', Rest)

Peter