[mercury-users] ... Still compiling fact module ... is assoc-list access fast?

[Douglas Auclair]

Dear all,

I'm working on a system that uses large amounts of data (data sets are 
usually 1 meg but can be 20 meg stored as line-delimited text).  Since 
scanning all this data into memory does have a start-up cost (on the order 
of seconds) and the system I'm designing is stateless (so it will be started 
up many times -- once for each inquiry), I've thought of compiling the data 
into code (it's static data sets).

This works fine for deterministic data sets[0], but for nondeterministic 
data the compiler dies (it runs out of memory rather quickly).  So I 
converted the nondeterministic data into semideterministic data (by 
returning lists instead of creating multiple facts), but it appears that the 
amount of data is so large that the compiler is still compiling the code 
after 7 hours.

Ouch!  Does this group see alternatives I'm missing?

I'm next going to convert the large set of facts to an association list 
(alist) and do look-ups that way (I tried compiling an alist before ... it 
took a while, as well).  This begs the question:  which is faster at 
runtime, a semideterminstic fact lookup or an alist search (lookup)?[1]

I must say I'm a bit surprised at the prolonged compile time.  There are 
only ~6500 semideterministic facts for one predicate declaration (~6500 
nondeterministic facts crashed the compiler) that have an int in parameter 
and a list out parameter (the list is a list of a type that is composed of a 
float, a string, and a float).  The total size of the source code of that 
module is "only" ~850K.  Am I doing something terribly wrong?

On the other hand, the compiler is very fast with a fact_table:  I like how 
the fact_table pragma can very quickly make 10 megs of facts accessible to 
the system.  What is the level of difficulty in extending it for other 
types, like lists and tuples?  Is there something in the compiler that can 
be improved to speed up the compile times for very large lists, alists, and 
facts?

Sincerely,
Doug Auclair

[0] I use the :- pragma fact_table on the data formatted as Mercury facts.
[1] This question is not, however, a determining factor in my design ... I'm 
sure both are very much faster than reading in a file at start-up.  I'm 
asking the above question more out of curiosity.


_________________________________________________________________
Help STOP SPAM with the new MSN 8 and get 2 months FREE*  
http://join.msn.com/?page=features/junkmail
--------------------------------------------------------------------------
mercury-users mailing list
post:  mercury-users at cs.mu.oz.au
administrative address: owner-mercury-users at cs.mu.oz.au
unsubscribe: Address: mercury-users-request at cs.mu.oz.au Message: unsubscribe
subscribe:   Address: mercury-users-request at cs.mu.oz.au Message: subscribe
--------------------------------------------------------------------------