Older blog entries for tbmoore (starting at number 3)

I've set the garbage collector work aside for a week and have been hacking on another home project. I've written a C header file parser for Common Lisp. This code digests header files that have been run through the C preprocessor and parses the subset of C that is likely to be in a header file i.e., most things other than function bodies. Constant expressions are supported. A parse tree, with each element of C grammer represented as a CLOS object, is produced. I've also written some code that walks that parse tree and emits the def-alien declarations for CMU CommonLisp. So, maybe we're a little bit closer to automatically interfacing with C? Maybe. I hope to make an initial release by the end of the week.

dan: Yup, this is the generational, conservative garbage collector I'm talking about. Enabling gencgc_verbose = 2 is very enlightening; you see what crap gets tenured, especially during compiles, only to later be scavenged.

I'd like to get all the ports of CMUCL to use this collector; there's no advantage of the old stop/copy collector that I can see, other than simplicity and possible lack of support for write protecting pages on some platforms.

So, last night I did some hacking on CMUCL's garbage collector, as advertised. The work I've done is pretty minimal at this point: just adding some additional statistics such as time spent collecting, page faults incurred, etc. I have a plan to tune up the garbage collector and these are the first baby steps in that direction. I think it'd be cool to let the user choose one of several GC policies: fast (generally short pauses), real time (very short pauses, 5-8ms in general), throughput (reasonable pauses but minimizing the total time spent collecting rather than the length of any one GC), and a spiffy auto GC policy that figures out which policy would be best for the current session.

CMUCL's garbage collector is written in C, and I was somewhat resigned to doing this tuning work in C too. Last night I had an "Aha!" about this: don't implement the policy code in C, which would be a drag, but export statistics and tunable parameters from the C implementation into Lisp. Lisp code that runs in an after-gc-hook then analyzes the statistics from previous GCs and diddles the parameters appropriately. Much better; after all, if I wanted to program in C I'd be hacking on the Gimp or something :)

A motivating factor for going in this direction is the fact that working on CMUCL's C code is a pain in the rear. CMUCL is a combination of a small C/assembly program providing some primitive services (such as memory allocation and garbage collection) and a much larger image of Lisp code and data. The Lisp code knows the addresses of important routines in the C code so they can be called from Lisp, but when the C code changes the addresses of those functions change too and the Lisp image has to be rebuilt. How that's done is a subject for another time, but it's somewhat more involved than just typing "make".

Just signed up; wheee! Now I'm going to go home and hack on CMUCL's garbage collector for a while.

New Advogato Features

New HTML Parser: The long-awaited libxml2 based HTML parser code is live. It needs further work but already handles most markup better than the original parser.

Keep up with the latest Advogato features by reading the Advogato status blog.

If you're a C programmer with some spare time, take a look at the mod_virgule project page and help us with one of the tasks on the ToDo list!