Older blog entries for crhodes (starting at number 123)

It's been a while. Lots of interesting[*] stuff to talk about. Since my previous diary entry was about my work for the European Lisp Workshop, why not start with my recollections and observations about that?

The workshop itself was one day long, and had one invited talk, five papers presented (one of which was mine) and a more informal demo / presentation of current advances in ContextL (a Lisp implementation of Context-Oriented Programming), given by Pascal Costanza.

The invited talk was given by Alexander Repenning, and was titled “Antiobjects” (and subtitled “Mapping Game AI to Massively Parallel Architectures using Collaborative Diffusion”, for what that's worth). The central idea, I think, was to convert a goal-oriented architecture – expressing behaviour of agents in the system in terms of what their goals are, in what is maybe the typical object-oriented programming style – into an optimization problem, viewing the system as a whole.

This change of perspective (the “Antiobjects” of the title allows for some interesting emergent properties. The compelling example Alexander gave was for ghosts chasing Pacman: the difficult bit in the traditional object-oriented style is in implementing collaborative strategies: if there are two ways to reach Pacman, and two ghosts, you (the evil games programmer) would like the two ghosts to take one way each. Generalising for large values of “two” is tricky. The Antiobject solution is instead to define a Pacman field, which diffuses away from the Pacman source, and have the ghosts be sinks of that field. Then the ghosts simply need to do hill-climbing of the Pacman field, and the desired behaviour of collaborative-Pacman-eating appears as if by magic.

Sébastien Mosser gave a talk about meta-models which, I have to confess, I didn't understand at all. I don't know what a “model” is, let along a “meta-model”, and unfortunately Sébastien didn't really give me an idea why I should care. I welcome enlightened discussion on this, if I should in fact care. The next presentation, reporting work by Pierre and Simon Thierry, was a general introduction to transparent persistence, which (as Pierre said) is almost a textbook example of using the Metaobject Protocol. The picture is slightly clouded because unfortunately update-instance-for-redefined-class is not quite able to cope with multiple class redefinitions in between accesses to instances. I believe I have a solution for this issue, but the margin of this blog is too small to contain it. (In practice, people work around the problem without too much pain: I discussed this at the workshop with not only Pierre, but also Arthur Lemmens of rucksack fame.

After lunch, I gave my talk about user-extensible specializers, details about their implementation, and examples of their use. In the course of giving examples, I inevitably made the mathematician's mistake of attempting to persuade the audience that simplifying (* _x 0) to _x was a good idea, but apart from that I don't think I embarrassed myself too much. What's interesting is that Jim Newton's talk, scheduled immediately after mine, was about an application of user-defined specializers, though in the SKILL (or maybe SKILL++, I forget) language of Cadence rather than in Common Lisp. (For the Common Lisper who is intrigued at seeing an application for user-defined specializer classes, as well as reading Jim Newton's paper I can recommend getting my implementation of the ideas by doing darcs get http://common-lisp.net/~crhodes/vclos, and playing around with the source files and examples therein.

After that, Nikodemus talked about his work on making the various caches in SBCL's implementation of CLOS thread-safe; we got to see some current directions in Context-Oriented Programming development (as well as news that a very few people are actually using it in the wild!), and then it was off to dinner, where I manage to surprise Edi by telling him that his webserver was too complicated, and David managed (not entirely) to surprise me by not talking at all about putative CLIM backends...

[*]In My Opinion.

People* like to say that Common Lisp is the programmable programming language, or the #1=(programmable . #1#) programming language: possibly because much of what Common Lisp provides is in some way malleable: the reader, the printer, the pretty-printer, the object system and the syntax are all to a large extent customizeable, through *readtable*, print-object, *print-pprint-dispatch*, the Metaobject Protocol and macros respectively.

Not everything in CL is in such a marvellous state of perfect adaptability – though perfect adaptability does not necessarily mean unlimited customizeability, as there are concerns about being able to compile to efficient code where that is desired as well as being able to mold the system to meet a set of needs. I've already discussed one case of (by default) non-extensibility, that of subclassing the sequence System Class; Common Lisp has a wide range of functionality that is applicable to sequences, but no way of defining new kinds of sequence, instead restricting the applicability of those sequence functions to just singly-linked lists and vectors. In this modern age, that's a little bit unambitious, and so I presented a paper at the International Lisp Conference this year about how to extend CL to allow the user to define his own sequence classes.

For the 4th European Lisp Workshop (in conjunction with ECOOP), I've looked at a similar example of user-extensibility, though maybe a little more esoteric: examining the ability of the user to define subclasses of the specializer metaobject class (and how to use those new specializers in the existing CLOS metaobject protocols). The first draft of the writeup around this issue is available; the document linked there is not final, but it is correct in the broad brushstroke summary, which is that in this area the Metaobject Protocol is poorly implemented (an even more executive summary might be: get SBCL 1.0.7 if you want to try it out...)

[*] some people, anyway. It's possible for people to get caught up in hype and to make claims that they cannot themselves justify, appealing instead to some higher authority when challenged; one common pattern with respect to Common Lisp could be called the brucio pattern.

More Gsharp blogging, I'm afraid; we will return to pure Lisp implementation hacking in a future edition. Until then, I'll mention that magnusjonsson has showed up out of the blue, with an avowed interest in providing support for microtonal mutations (in the form of notation and performance of sagittal accidentals, which are apparently good enough for the microtone community). I think this is awesomely cool, and it even potentially ties into my own musical research interests, such as whether one can infer the tuning system required for best performance of certain Renaissance repertoires (e.g. syntonic tuning in Willaert motets).

Relatedly, I recently discovered that I work (at one remove in management structures) with a developer on a different (competing! or maybe that's giving Gsharp a little too much credibility) music notation programme. My hope is that I stay around in my current position for long enough that interesting work in that space can happen, because in neither of our cases are we in fact employed to work on music notation – even though it is extremely close conceptually to the things that we are paid to do...

Continuing the theme of being guilt-tripped into making Gsharp tolerable to use (rather than just pretty in screenshots...): over the weekend I implemented selecting the active layer in a DWIM-like manner for the common case, moving up and down between staves using C-up and C-down (and then implicitly selecting the first layer on that staff).

Additionally, I implemented a hack so that the cursor is always visible in the viewport. It's a hack, because (despite the neat double-buffering support in McCLIM), there is visible flicker in some circumstances. The reason is that the viewport positioning is done at the wrong time – after the score has been drawn to the pane. Fixing this would involve more work than I was prepared to commit to on a lazy Saturday afternoon, though.

We're close two working weeks into Google's Summer of Code, and the student I'm mentoring, Brian Gruber, is making good progress in his project to add MusicXML support to Gsharp – sufficiently good to blog about it, in fact.

Discussing things with Brian, and fixing some of the embarrassing bugs in gsharp that he found, has also encouraged me to tidy up a little some of my accumulated divergences from the Gsharp CVS and commit them; in particular, the lack of line- and page-based motion commands was a significant annoyance last time I edited a score in Gsharp, so I spent an evening working out a sufficiently good implementation, and now C-a, C-e, C-x [ and C-x ] work as an emacs user might expect. (It's ever so slightly trickier than it might seem, because there's no information in the gsharp buffer recording layout information per se; the gsharp buffer contains a complicated data structure representing each individual musical (or graphical) object, but the page and line layout is computed separately – so motion by layout element needs additional computation).

Other things coming soon to a CVS repository near common-lisp.net, to whom thanks go for their hosting: correct placement of whole-bar rests (in the middle of a bar, if they are the only element in their layer in that bar); a hacky printing facility; maybe even a reworking of the currently-useless panes on the right-hand side of a gsharp window...

In my previous diary entry, I somewhat sarcastically obeyed the letter of the BSDish-with-advertising licence of cl-pdf. Since then, Marc Battyani wrote to me to say that he was removing the advertising clause from the licence of cl-pdf (and cl-typesetting), replacing it with a request (not part of the licence) to acknowledge its use in significant applications, to assist in promoting the Common Lisp language. Thanks, Marc, both for the software and for the friendlier licence.

Some people have expressed mild interest in the code used to produce the name badges for ILC 2007. I used cl-pdf to generate PDF, and otherwise there's only a very little bit of cleverness to get nicely filled and indented lists of features and similar. Feel free to download it and use in whatever way you wish, subject to the licence of cl-pdf (which seems to have an advertising clause, so I must say of defperson.lisp that This product includes cl-pdf for generating documents.)

As I wrote in my previous entry on ILC 2007, I was away from the conference on Tuesday. From talking to people whose judgment I trust, it appears that I missed some spectacularly awful stuff; however, now is probably not the time to comment further publically on such matters. Of the good stuff, I was most sorry to miss Richard Jones talking on Garbage Collection; having the book and the paper that Richard wrote for the proceedings aren't a substitute. Apparently, Charlotte Herzeel made some people begin to see some point in Aspect-Oriented programming, too, which is no mean achievement given that the usual response is that it's just one sufficiently-advanced method combination (well, it is, but it's also using it pervasively).

Anyway, Wednesday beckoned. The start at an ungodly hour was redeemed by Manuel Serrano's demonstration of HOP, an environment for programming the Web. I don't really live in the Web programming sphere; it doesn't interest me a great deal: but the presentation was excellent, with well-timed automatic slide changes, fun demos, and a clear explanation of the problem space; I don't know whether having the need for a proxy server will render it difficult to deploy in practice. Good stuff anyway; in fact, it worries me ever so slightly that the best of the invited speakers for me – Sperber, Queinnec and Serrano – are very firmly in the Scheme camp.

After that talk, I found speakers failing to command my attention quite as much as the impending delivery of my own, about extensible sequences; I did enjoy Geoff Wozniak's about dynamic data types, defining a protocol whereby instances would be able to change their own implementation details based on runtime profiling data; I think I need to look at the details of his ideas.

Then the time came for me to stand up, and – wouldn't you know it? – the laptop refused to display simultaneously on screen and on projector. Which is ridiculous, because it was fine on Monday for Herbert Stoyan's talk. I bravely soldiered on without the comfort blanket of my notes in front of me, and I don't think that any problems in the reception of my talk were particularly because of that. In retrospect, the example sequence class I used in the talk was too simple, and I didn't give compelling-enough other concrete examples; fortunately, I was able to make up for some of that in the coffee break afterwards, talking to a couple of people (and Cyrus did a sterling job talking to Jans Aasman about his space-efficient implementation of strings of DNA, and Jans then started talking about treating memory-mapped files as sequences, so I think he probably got the idea).

Then it was all over bar the Liskell talk (not particularly interested, though good on Clemens for having the gumption to stand up there and tell it like he sees it), the Franz programming-in-the-large talk (Jans is a fun speaker; I thought that Luke asked a rather good question at the end about the expected reliability of garbage collection), and a little bit (fortunately, not a lot) of self-congratulation at the end. Dinner at the Hotel Felix with, among others, an exhausted local organizer wrapped up the day, and I was homeward-bound twelve hours later.

So, that was Monday. (The observant will realise that this is Tuesday^WWednesday; since I'm not at ILC on Tuesday this still counts as being on time). I got asked^Wsuckered into chairing the first session, which inevitably involved faffing around with projectors and broken laptops, and I didn't get time to eat the croissant that I had thoughtfully purchased on the way in to the Law faculty. So, I chaired with a rod of iron, so that we could get to lunch on time: but Clare staff weren't ready for us. Grr.

Oh, the talks? Well, I'm probably not the best judge of the early talks; a breakfastless chair is not a happy one, but also I couldn't see the slides from where I was sitting. Herbert Stoyan's discussion about the early days of Lisp history was fun; I was less excited about the source to source semantics-preserving compilation that he discussed – I didn't really understand why it was relevant. This was actually a common theme in the talks: the speakers rarely succeeded in explaining what the problem that they were addressing was. So Stephan Frank didn't really explain why constraint propagation was important or interesting; I'm not a screamer user, so comparisons involving it ("it's like screamer, but faster") didn't really help. Still, when he's demonstrating benchmarks, it's nice that the fact that he used SBCL is accepted as a matter of course.

Similarly, the Dylan DSL talk (by Hannes Menhert and Andreas Bogk) didn't really state up front that part of what they were addressing was the lack of procedural macros in Dylan. They got to it eventually ("we rely on the compiler to constant-fold this 1+1+1+1+1+1+1+1+1+...") but if you hadn't read the paper in advance, that might not be obvious. The demo was good, but I found the rest of the talk a little bit disappointing: lots of explicit examples of macros which would be a lot easier to write in Common Lisp. Hey ho.

Continuing the theme of non-explanation, we got a talk on FREEDIUS, a reimplementation of 3DIUS funded by SRI. We had to infer that the interesting stuff was nifty 3d scene reconstruction and object detection and interaction; obviously, the demos made that rather clearer. It was somewhat heartening to see that McCLIM is not the only thing that suffers from redisplay glitches: the demos indicated that there's clearly an OpenGL (or similar) buffer being held on to for slightly too long.

Then we broke for lunch, and I was allowed to give up my chair. Hooray. We had an ESA talk, which I basically knew about; an LTk talk which I'd basically heard the entirety of in Hamburg, and a talk about gene patterns and pixel-based clustering which was cool and new; it's a bit of a shame that Cyrus ran out of time to talk about the lisp-side stuff, but on the other hand I am of the opinion that too much "this is lisp, isn't it amazing" is bad for the brain.

Then we discovered that Antonio Leitão had not arrived. So we had a long coffee break. Awesome! Also awesome was Christian Queinnec's talk about teaching Scheme to undergraduates (800 at a time: there were some very interesting effects resulting from automated grading systems, including an equalisation between the sexes' performance, in UMPC's implementation at least), and Michael Sperber's talk about R6RS was good too: clearly the editors have suffered in the name of standardization...

And then I had to head off to London, only returning in time for Tuesday's banquet. Which was excellent, and great fun. Now all I have to do before tomorrow is to write my own presentation. (And get some sleep.)

ILC 2007. Hooray, it's actually international—that is, it's not in the USA. In fact it is in Cambridge (the real one), and it is going swimmingly. Well. It's certainly nice to have of the order of 120 interesting people around; there's a good breadth and depth to the field, what with academics, hobbyists, and the occasional person who uses lisp for Real Work: sometimes even SBCL explicitly. The organization has been impeccable so far, so well done Ravenbrook, and particularly Nick Levine.

Today (Sunday) was tutorial day; I have to say, I found the tutorials disappointing. In the morning, I went to the Memory Pool System tutorial. While I couldn't resist a snort of derision at the news of the failure of MPS/CMUCL integration ("we couldn't work out how to build it with trivial source changes, let alone a new allocator"), I was more disappointed with the disorganization of the tutorial—which was probably exacerbated by the tendency of some in the audience to indulge in reminiscences and anecdotes about development avenues of the 1980s. There was some interesting stuff: I was particularly impressed by the notion of reference rank, and I also need to think more about the five-phase collection algorithm. Still, I spoke briefly afterwards with Richard Brooksby, and I think I know what to do to help them to move forward with their ideas.

In the afternoon, I started off by going to the tutorial about Debugging and Optimizing in Allegro CL; however, I confess I didn't get very much out of it: I already know what it's possible to do with a bit of assembly (or pseudo-assembly), and it doesn't really look like they are so far ahead of other ways of doing things. There are only so many ways you can convince gdb to do something useful, for instance. So for the second half of the afternoon I went to Pascal Costanza's tutorial about ContextL, which was at least challenging and thought-provoking. (And Pascal had clearly done enough preparation not to be left floundering...)