[m-users.] Crash in improved ODBC library when compiled in hlc.par.gc grade

[Fabrice Nicol]

Hi Volker,

I have not reproduced the failure, nor did I closely inspect the relevant
code. This said, I met somewhat comparable issues 3 years ago in work that
I unfortunately cannot open source.

To boil down the lessons I drew to the bare essentials: success in one
grade does not establish that the C interface is correct.

Plausible causes include an invalid ODBC buffer length or lifetime, an
out-of-bounds write, a Mercury pointer retained in storage not visible to
GC (see 4-5. below), incorrect heap-state handling across C/Mercury
boundaries or, more probably, concerning parallel runs, a race involving
shared handles or buffers.

And the above does not preclude the possibility of a compiler/runtime
problem.

I would perhaps try the following:

1. Make a small standalone reproducer with fixed database contents, query
and ordering. Include a worker count. Cleanly rebuild all Mercury
components for each grade. Compare hlc.par.gc with hlc.gc where supported,
and vary worker count separately; repeat the original asm_fast comparison
with the same compiler.

2. Audit every allocation path around MR_make_aligned_string_copy,
including C helpers and callbacks.

3. Check source-buffer termination & lifetime (if you can determine it
robustly) and how returned Mercury strings remain reachable (under thevsame
provision). This may be the hardest thing to do for parallel grades, see
why below 5.

3. Build with C debug symbols and capture the SIGSEGV in gdb (or a core
dump), including "thread apply all bt full".  Log row number, column,
buffer capacity and reported length immediately around fetch/copy
operations.  Use AddressSanitizer/UBSan to seek the first invalid access.

This requires some GDB know-how but knowing your coding mastery, Volker, I
assume this to be entirely feasible by you.

Abstracting away from the use case at issue, my prior comparable experience
and a subsequent plunge into Mercury's memory management details have led
me to the following conclusions, which admittedly are somewhat subjective
and so entirely debatable:

4.  When you investigate how Prologs historically managed memory, you
discover that memory was initially managed without GC. GNU Prolog
preallocates memory in excess of actual needs and still does not use GC.
Scryer Prolog was created using the same (primitive) memory management
pattern. There is a whole fascinating (though somewhat technically
outdated) book on the issue about older Prologs, which I am afraid still is
entirely relevant (Memory Performance of Prolog Architectures, E. Tick,
Kluwer, 1988 — should be an obligatory must-read to any LP student or
serious practitioner).

While Mercury has its own memory management system, there remain common
issues that are pervasive in all LP programming languages, and make it all
but impossible to align with Rust-like management of memory chunk
ownership and lifetime (see Zoltan's reply to a prior thread I initiated a
few months ago).

5. While there is some documentation in the repository on how GC memory
allocation is implemented, I still am at a loss to understand how it
interacts with parallelism, which is a late move in Mercury and I am not
sure all issues have been tackled so that, in parallel grades/runs, memory
(de-)allocation, access to pointers and GC work seamlessly.

If ever GDB does not help you out in steps 1-3, there might be a clue in
4-5.

Best,
Fabrice
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.mercurylang.org/archives/users/attachments/20261019/92c48783/attachment.html>