Older blog entries for crhodes (starting at number 84)

chalst says

For what it's worth, an outsiders view of what Common LISP needs[...]

The problem with this is that one thing that's worse than an insider's view of what Common Lisp needs is an outsider's view.

Why should that be? Well, the goal of evolving Common Lisp isn't, or shouldn't be, to turn it into something which looks and quacks just like the fashionable duck of the day. So a Scheme adept might bemoan the lack of full multi-shot re-entrant continuations, while ignoring (or being ignorant of) the fact that requiring such effectively kills off any hope for compilation to efficient code. An ML wizard might want greater support for typechecking; a C++ expert might hope for greater templating. All of these are solutions looking for problems in the context of Common Lisp: the lack of multi-shot re-entrant continuations, templates or statically-typed Lisps is not stopping people from getting work done, and there is no demand from people currently using Common Lisp for these features.

Would adding these features instead encourage people from other communities to leap on the Common Lisp bandwagon? I think that's unlikely: why accept Common Lisp with Schemey bits tacked on if you already have Scheme? Only if CL provides you with things that Scheme does not and cannot will that work: and let us not forget that implementing these abstractly-requested features would take time, and lots of it — probably enough for R6RS to appear.

In one respect, in any case, comparison with SRFIs is misleading: Scheme needed the SRFIs to allow people to decide on a shared vocabulary for completely ordinary programs, for the utility functions that previously everyone was having to implement for themselves: it's not hard to write remove-if-not, but if everyone has to do it then that's so much less time to spend on solving the actual problem. In other respects, of course, the SRFIs are doing useful work, but a brief perusal seems to indicate that most of the implemented SRFIs are essentially replicating the standard Common Lisp “library” (for want of a better word).

So what should CLRFIs be doing, then? Well, if I had infinite free time I might write an external-format CLRFI, to harmonize treatment of different character sets when talking to the outside world. A networking CLRFI might be nice, but I think it's probably still not clear how to make it sufficiently platform-independent as to allow natural implementation everywhere CL already exists; it might be better to attempt to standardize higher-level networking (at the protocol level) and allow implementations to use whichever socket layer is to hand. I've already discussed chalst's desire for continuations: there are portable libraries for achieving single-shot continuations, or macros enabling a natural expression of continuation-passing style, but requiring implementations to provide native full first-class continuations is a non-starter. Hygenic macros in Common Lisp would be a solution to a non-problem: given the years of experience in programming in the language (not my years alone: other people's years too) it is clear that the lack of hygiene in the macro system is simply not an issue to the Lisp community. As for #! support, it's really not clear to me why that should need to be available in a standard fashion: given that it's clearly a deployment rather than a development issue, by that stage an implementation choice will have been made, and one can simply use that implementation's facility.

That leaves, from chalst's wishlist, an FFI (where the "F" there actually means "C") standard. And here there is definitely a case for wanting firmer ground to stand on: people using the language at this very moment would prefer the various Lisp vendors to get over their issues with each other and just coöperate. The good news is that it's not all bad news: UFFI is a “user-space” solution to some of those needs (though not all, because by its nature it supports the intersection of available features). A good first step might be a survey of the different implementation's provisions with a view to establishing a reasonable union of features to request from the vendors, and then applying pressure or submitting patches. (Which reminds me: when I have enough free time, unless someone does it before me, I must look at the user-contributed suppport for callbacks on the PPC... time is the key).

Here's a less silly screenshot reflecting the progress of SBCL's upcoming Unicode support. A limited number of external formats have been implemented: enough to act as proof of concept, as demonstrated there. The TODO file is beginning to read more like notes of the process we went through rather than the mountain that has yet to be climbed, which is definitely cheering.

In other news, I went to Amsterdam last weekend for a local lispmeet (well, it was flagged as benelux, but when you're getting people from the UK and Sweden, it's looking more than just that.) There was much of interest, from Peter van Eynde's stealth introduction of Lisp resulting in him being “forced” to do Unix systems engineering in his favourite language, through Edi Weitz' rdnzl .net/Lisp bridge, to the ever-present question of evolution of Lisp: this time Nick Levine represented the clrfi-editors (those who know Scheme will spot the similarities with the SRFIs) with a plea for a high-quality submission to see if their process could be unblocked.

On this last, my view is that the Lisp community, whatever that is, certainly needs a process to evolve the language (preferably backwards-compatibly, but some things might need to be removed root-and-branch), it also needs people to do the work to produce high-quality standardization material: and it's more the lack of that that's lacking, and no amount of process will generate it. I'd love to be proved wrong.

SBCL Unicode work is progressing well. It's nearly at the stage where it becomes useful: it's already at the stage where we can do silly demos such as in this screenshot. Clearly, this isn't going to make it into sbcl 0.8.15; however, I have hopes that it will into 0.8.16 (or should that be 0.9.0 or even 0.1point0.pre1? We still have a little work to do before everybody's happy with supporting a 1.0 release: the issue isn't so much the stability of the code (which was engineered to last — parts of it are decades old!) as the desirability of supporting a suboptimal interface as stable; the main benefit from a developer's point of view of a pre-1.0 release is the ability to laugh away complaints about lack of backwards compatibility. Such days may soon be behind us.

We live in interesting times. SBCL has acquired a new web page layout (with some extra content, too!); Nathan has almost completed the 64-bit work which I dropped after submitting my thesis; Nikodemus, when he returns from studying hard, is likely to give us linkage tables (you can solve everything in Computer Science with an extra layer of indirection) and a single stepper; and I, along with some friendly helpers (and armed with four years of discussions on #lisp IRC), have started the slow process of making SBCL Unicode-aware.

With luck, people will use these exciting new features. It's sometimes hard to tell whether one is programming into a vacuum: I count myself lucky that I can work on SBCL projects for intellectual satisfaction, though I confess that I do prefer it when there is visible positive feedback from actual users...

Edinburgh was fun. Hard work, but fun. Singing with a nice bunch of people is always a good thing, and when we're doing such challenging masterworks as Tallis' Spem in alium (where everyone has to pull their weight: it's for eight choirs of five parts each...) it's good for the brain cells too: certainly better for them than the post-concert social occasions, aided and abetted by Scotland's sane licensing laws — unlike in England, alcohol may be served after 11:00pm.

Juho Snellman is a smart cookie.

SBCL is a large, complex beast: I've talked enough about how hard it is for a system to give the right answer, but when people start wanting it to give the right answer quickly, well... Juho Snellman has single-handedly found improvements to string hashing routines, to poor scaling behaviour with multiple threads, and now bignum division. It appears not to solve the performance problems that we're having with GSharp (which are probably algorithmic in nature: why are we doing arbitrary-precision calculations, anyway), but I should hope that some of the graphs in Andreas Fuchs' boinkmarks (once the nightly run has completed) will look interesting.

I've put up version one of proceedings of the lightning (5-minute in theory, closer to 10-minute in practice) talks from the Bordeaux Libre Software Meeting Lisp track. Not everyone made my arbitrary deadline, but most did: and for those who didn't, there is the possible lifeline of a version two at some point in the future. (I hope we'll get Gilbert Baumann's thoughts on reader/editor integration, and Tim Moore's on CLIM interoperability: it was interesting stuff).

Comments and corrections are welcome: either to the e-mail addresses listed therein, or on freenode's #lisp channel, where quite a lot of the authors hang out.

As for my “keynote” talk, which I'm conscious that I haven't discussed yet: essentially, I attempted to make the case for protocol-oriented programming.

“What is protocol-oriented programming?” the cry rang out. Well, it probably already has a technical meaning (I confess: I didn't check), but the way I mean it is as at least cursory thinking about whether it makes sense to document the means by which a program (or function, or basic computational unit) achieves something, as well as what it is intended to achieve.

Why would one want to do that? Well, in languages less impoverished than C and its cousins, it is possible to hook into provided functionality: Lisp in particular is very trusting, allowing the user (or, perhaps the author of library Y) to override behaviour in library X. A protocol is then the contract between these two libraries (and any overriding that has occurred) so that the resulting mixture continues to function as expected.

An obvious example of a protocol in this sense is the CLOS Metaobject Protocol: libraries or users are allowed to selectively override slot-value-using-class, but if it's overridden inconsistently with (setf slot-value-using-class), expect Bad Things to happen (so Don't Do That Then). However, I believe that there's much greater scope for protocol-oriented programming than has hitherto been realized: for instance, I bet the McCLIM people would love a protocol for fontconfig, rather than an opaque library. And so on, and so on.

Part of the point I was making was that the Lisp community is too small to build isolationist libraries, while undisciplined development (first, take a copy of the library; then, modify the source) leads to a scaling problem and a combinatorial explosion, making it difficult for any given user to use the enhancements by two other people. Of course, the downside is that protocol-oriented programming — or exposing your internals (see, it even sounds disgusting) — is difficult. Film at 11.

Bordeaux was fun. Let's do the amusing local colour things first, to suck you into reading about deeper, more meaningful stuff later.

Bordeaux has a waiting school (that's a school for waiters, obviously, not an institution that's anticipating some event). My visits to Bordeaux in the past have seen some of the products of this school; apparently, this time we met all the drop-outs. Star of the show was the gormless waiter at the pizza restaurant in the north of the city, who was incapable of taking the order (really! He had to get the chef out of the kitchen to take it), and ran out of supplies — these things happen; it is possible for restaurants not to have deliveries of things that not everyone wants, such as beer, but the limited supply of tomato sauce in a pizza restaurant is close to unforgivable.

The new tramway system is entertaining, too. Apparently rushed into production for political reasons, it's effectively undergoing live testing. When it works, it's relatively efficient; central Bordeaux to the ENSEIRB in about 25 minutes isn't too bad. For three of the eight journeys I took, that's more-or-less what happened, though in one of those three we did wait 30 minutes for the tram to arrive; the other five suffered breakdowns of some sort. Most amusing was the time we were warned that the line was down in two stops, and we would have to take a replacement shuttle bus; unfortunately, our tram broke down before we got there...

As for the conference itself. I was suitably impressed by the Smalltalk whizz-bang demos; it is nice to have pretty stuff on the screen. My talk (about protocol-oriented programming, on which more some other day, I think) was rather prosaic by comparison: just simple acclaim slides talking about various forms of extensibility, and why it might be a good thing. Much more interesting from my point of view were the lightning presentations: quick five-minute talks about things various people found interesting. Rudi Schlatte on simple-streams and Eric Marsden on serve-event were particularly entertaining, and Bruno Haible on case-preserving packages gives a different take on so-called modern mode lisp. I'm attempting to collect short summaries of these from the speakers to make proceedings available: stay tuned.

... and we're back! Inevitably, by the law of Professor Murphy, just as I'm about to leave for the Libre Software Meeting; also, by Parkinson's corollary, I've forgotten all the bloggable stuff that I could write here. Go look at this (scroll down to the end) for the executive summary.

One thing: I attended (and gave a paper [slides] at) the 1st European Lisp/Scheme Workshop in Oslo. That was fun; meeting new people, putting names to IRC nicks, and so on.

And now, I'm really looking forward to going to the hothouse of Lisp programming that is Bordeaux: as well as a good list of attendees, the locals (Robert Strandh, Tim Moore, Iban Hatchondo, Matthieu Villeneuve) should provide good conversation too. Particularly looking forward to the lightning presentations; less looking forward to finishing my talk...

À bientôt!

Something else that happened in the dark ages of the diary: I went to scheme-uk again. This should not be taken as indicative of any paradigm shift towards the Dark Side; however, a chance to hear Shriram Krishnamurthi speak is not to be sniffed at.

So what did he speak about? Shriram actually gave two talks: the second was probably more audience-pleasing (in that, well, it involved scheme: specifically DrScheme and FrTime), on the beginnings of functional GUI programming. Neat, but time will tell whether it's a practical advance on the current callback-based paradigm.

The first talk was, serendipitously, closely related to work I've been doing: Shriram presented results of examining the coverage of deltas (individual CVS checkins, say) by test suites, coming up with the mildly surprising result that the coverage is strongly bimodal: most changes affect either very few tests (one or two) or the whole suite.

This is similar in method and scope to work that I'll be presenting in Oslo at the Lisp/Scheme Workshop (side note: only €25 for attendance) on classifying benchmarks by examining the changes in timings caused by source deltas. Shriram's talk also leads naturally into repeating his analysis using pfdietz's test suite, with the added advantage that it's cross-implementation, and so not contaminated by a biassed worldview. On the other hand, the ANSI CL test suite isn't finished yet; this might cause a little difficulty in comparing results.

Still. Interesting stuff.