Older blog entries for crhodes (starting at number 98)

We released McCLIM 0.9.2 ("Laetare Sunday" – go silly names for releases!) and SBCL 0.9.11 at the weekend.

One relatively new feature in SBCL is the ability to make executable images in a fairly straightforward manner: simply pass :executable t to sb-ext:save-lisp-and-die, and the system will attempt to dump one file that plays relatively nice with the executable loader on your given system. Since it is fair to say that there are a large number of dependencies in McCLIM, and additionally a non-dependency that is in practice needed to get windows to display on the screen, it seemed reasonable to use this executable-generating functionality to distribute a binary that could be tried out before deciding whether or not to get the source to a fair number of software projects.

So, Andreas Fuchs duly stepped up to build a binary of mcclim-0.9.2 for x86/linux, and all seemed well. However, I remembered that some people had expressed an interest in a Mac OS X binary, so I went about building one of those[*]. And all seemed well; a few things had to be changed from Andreas' binary, mostly because SBCL on x86/linux supports threading, while on ppc/darwin it has to make do with select(). However, trying the binary revealed some odd things going on: using Apple's X11.app, you would need to hit RET twice to get your input accepted: only not quite systematically: also, sometimes, the REPL's answer would only appear after a subsequent keystroke.

Amusingly, the same behaviour was far less systematic when running the same binary over an ssh-forwarded connection to an xorg server. This is perhaps not overly surprising, but it did make debugging the problem, or even identifying the proximate cause, a lot harder. Tim Moore and Juho Snellman did end up tracking it down, though: Tim worked out that read-char-no-hang was hanging, and Juho spotted an error in a recent bugfix in SBCL for listen: sometimes the system would report that data was available on a socket even when it wasn't. The difference in behaviour of the buggy binary between X servers is I think explained by a difference in double-buffering: Apple's windows are double buffered, so they never receive any Expose events, whereas xorg's aren't, and so they receive many more events in total; and a failure to respond in a timely manner to an Expose event is less noticeable than not doing anything when hitting the Return key...

The upshot is that SBCL 0.9.11 is unfortunately unsuitable for McCLIM work; however, armed with a hotfix for listen, I have made available an OS X binary of mcclim 0.9.2 for people to try: you'll need X to be running, and also to have libfreetype.dylib in /usr/X11R6/lib/.

[*] For the record, the build script ended looking like this for OS X:

(mapcar #'require
        '(:asdf :clx :mcclim :mcclim-freetype
          :clim-listener :clouseau :swank :clim-examples :functional-geometry))
(load "/isms/home/mas01cr/lisp/cvs/clnet/mcclim/Apps/Debugger/clim-debugger.lisp")
;;; listener fixes / demo commands
(clim:define-command-table clim-listener::demo-commands)
(clim:define-command (clim-listener::com-plot-fishes
                      :name "Plot Fishes with Functional Geometry"
                      :command-table clim-listener::demo-commands :menu t
                      :provide-output-destination-keyword t)
    ()
  (functional-geometry::clim-plot functional-geometry::*fishes*))
(clim:define-command (clim-listener::com-show-demos
                      :name "Show McCLIM Demos"
                      :command-table clim-listener::demo-commands :menu t
                      :provide-output-destination-keyword t)
    ()
  (clim:run-frame-top-level
   (clim:make-application-frame 'clim-demo::demodemo)))
(clim:make-command-table
 'clim-listener::listener
 :inherit-from '(clim-listener::application-commands
                 clim-listener::lisp-commands
                 clim-listener::filesystem-commands
                 clim-listener::show-commands clim-listener::demo-commands)
 :menu '(("Application" :menu clim-listener::application-commands)
         ("Lisp"        :menu clim-listener::lisp-commands)
         ("Filesystem"  :menu clim-listener::filesystem-commands)
         ("Show"        :menu clim-listener::show-commands)
         ("Demos"       :menu clim-listener::demo-commands))
 :errorp nil)
(push (lambda ()
        (setf mcclim-freetype::*freetype-font-path*
              (merge-pathnames
               (make-pathname :directory '(:relative "fonts"))
               (make-pathname
                :directory (pathname-directory sb-ext:*core-pathname*))))
        (clim-listener::load-mime-types)
        (clim-listener::load-mailcaps))
      sb-ext:*init-hooks*)
(setf sb-ext:*invoke-debugger-hook* #'clim-debugger:debugger)
(sb-ext:save-lisp-and-die "/tmp/mcclim-listener-0.9.2/mcclim"
                          :executable t
                          :toplevel (lambda ()
                                      (clim-listener:run-listener)
                                      (sb-ext:quit :unix-status 0)))

If I have seemed less present, less immediately responsive these past couple of weeks: it is because I was tricked into having my nethack addiction rekindled. Fortunately, I can report that I achieved my first ascension this evening, and am therefore better able to concentrate on Free Software matters henceforth, until the next serotonin-producing arrangement of punctuation comes along. Lucky I never really programmed in Perl, eh?

Bother. I was going to do a before-and-after comparison of rendering of ties in the Gsharp score editor: before, from when I first implemented ties, and after, when Robert reimplemented them, including the beginnings of support for tie end glyphs, and for adjustment of tie heights for varying widths between tied notes. However, I managed to use the same file name for the latest screenshot as I did for the one almost two years ago (well, yes, ties.png is of course the obvious name for this: so obvious I chose it twice...), so you will just have to take my word for it that the new version is prettier than my hacked up version which just drew lines.

Instead I will just have to provide more evidence of prettiness, and since I have a long-held fondness for the music of Clément Janequin, I give you the beginning of les Cris de Paris, with the beginning of a lilypond version from about 2000 (in an evince window, behind) for a not-quite-fair comparison, and (what is more; see, how generous I am!) a MIDI file from Gsharp, just to prove that I'm not cheating.

Cyrus Harmon has been working on a port of the mostly-copying generational garbage collector in SBCL, based on previous work by Raymond Toy for CMUCL. At the seventh merge candidate, I decided I'd made him suffer enough with my tyrannical requests for whitespace changes and function renamings, so I merged it, having tested on multiple platforms, and went to sleep the sleep of the just.

Sadly, the testing wasn't quite exhaustive enough, and I discovered this morning that we'd managed to kill the overnight automatic build-and-benchmark host run by Andreas Fuchs. Fortunately Cyrus was awake (whether again or still I don't know; for him it was 0300 local time) and diagnosed the problem pretty quickly: we had managed to subvert the garbage collector almost, but not quite, completely on threaded platforms. Interestingly, the system was stable enough to build itself, but benchmarking it (or indeed running the regression tests) exercised different-enough allocation patterns that the existence of the error was pretty obvious.

A simple one-line fix later, and things started working on threaded platforms too.

John Wiseman wants a workaround installed in SBCL for a buggy, closed OS[*]. If that summary were all that there was to the issue, it certainly wouldn't merit a diary entry, even given the prominence of John's weblog among a section of the Lisp-using public; I don't think it hurts to discuss merits and faults of software out in the open, and of course no publicity is bad publicity.

However, John appears to misunderstand how people work. In particular, he wants to spur SBCL developers to fix the problem he observes for him[**]. Of course, this is a reasonable want, but as children are told, I want doesn't get, and this holds in the adult world too. Those who take out a support contract or similar arrangement have exchanged a quid or 500 for the quo that they then have a right to expect; those who don't must depend on the interests of those who do being aligned with their wants.

Now, here is where it gets interesting. There are all sorts of reasons why depending on the interests of others can work well in the world; after all, there's reasonable agreement over what is undesireable versus what is desireable; what is good vs what is bad. It's not Universal agreement, but it is there, and the more select the community the more likely it is that the agreement within that community is strong. So, for instance, it's clear that reporting a straightforward bug in a Free Software system, without providing a fix, is nevertheless likely to lead to the bug being fixed, as other individuals will weigh up the chance of being hit by the bug in the future against the effort required to fix it now.

What is different in this specific case, then? Why isn't this just a bug report? Well, as a cursory read of the relevant thread on the development mailing list would show, this issue was reported nine months ago, and no real attempt was made then to provide a workaround for the buggy OS vendor's tool. So this weblog posting isn't a bug report, it's an attempt to get other people to reprioritize what they are doing.

However, empirically, this has not bothered other people overmuch over the last nine months: no-one else has voiced a complaint, and no workaround has been written (obviously, since otherwise John would have nothing to complain about). So this might well be one of those cases where one's interests are not aligned closely enough with others; what to do, what to do? Well, one thing that John could do is to attempt to develop and install his own workaround for his situation.

John claims that he "... looked at trying to fix it [him]self, but as a first SBCL hacking project it was a little beyond [him]." This is certainly to be expected if he is deprived of all human contact, but here's where the interests of others can be put back in alignment: it is most certainly in the interests of developers of SBCL to have more people with enough understanding to be able to make non-trivial modifications to the source code. So while John may find that his attempt to spur developers to do his bidding is unsuccessful, he may also find that they are receptive, even without other consideration, to assisting him in solving his own issue.

The general point to make here is that, in at least this little corner of Free Software, where all the labour is volunteer, any user and developer community is an emergent property of individual and small-scale interactions; in this context, it is far from clear that making a loud complaint is productive for any party, whereas attempts to take personal responsibility, even if they ultimately need assistance, are more likely to bear fruit.

[*] All OSes are buggy. Not all of them prevent bug fixes.

[**] To his credit, he also claims to want to document the issue. Why he thinks a weblog post is better than the top hit that Google returns for "SBCL crash reporter", pointing to the mailing list discussion of nine months ago, I don't know.

[ Edited to fix a broken link ]

After a bit more fiddling around, I've merged Alastair Bridgewater's work (referred to in my previous diary entry) into HEAD of SBCL's CVS. This is far from an announcement of official support for SBCL on Windows; there is quite enough that doesn't work (see Alastair's TODO list or mine for ideas of what you could contribute) that interested parties need to fix before we could contemplate treating it as a stable platform.

Still, one step closer to destroying the software industry...

Windows can be made to look similar to Unix, from a suitable vantage point and given sufficient use of green-tinted glasses. (I watched The Muppets' Wizard of Oz yesterday, and oddly enough this seems like a good analogy: we're definitely not in Kansas any more).

The background: SBCL is a lisp environment (native-code compiler plus runtime) which is fairly solidly Unix-centric, and has been for the best part of 20 years or so; at its inception, it also ran on a Mach kernel. Among the Unixisms pervading the code are simple things such as parsing filenames; mild things like having streams associated with fds (rather than HANDLEs); and fairly fundamental differences in signals, exceptions, and handlers. This latter is important because a fair amount of runtime support is implemented in terms of Unix signals: the garbage collection is assisted by a write barrier; the debugger wants to be able to set breakpoints by writing an illegal instruction at the start of a function; and so on.

Why is this relevant? Well, I'm staring at a 65-page diff from Alastair Bridgewater, containing a sufficiently-complete port of SBCL to Windows. ("sufficiently-complete" here means that it self-hosts and mostly doesn't crash-and-burn instantly; it certainly does not mean that it's stable enough to run your mission-critical server on; whether you'd want to do that on Windows in any case I leave open.) This is the second iteration of this patch; Juho Snellman has already blogged about the first iteration, which was eight pages shorter; this was not the direction I hoped the patch would go, but oh well: I am assured that the extra eight pages include extra functionality, and certainly there is a lot less that causes screams of horror.

Alastair's made patch and binaries of the latest attempt available, as well as the beginnings of a TODO list: if there are Lisp-interested Windows hackers out there, more eyes and hands would be very welcome.

We have the beginnings of Climacs/PAIProlog integration. Code and screenshots soon. That is all.

Well, no, that is not at all "all". Work and multiferous other projects also continue to advance, some more glacially than others. Does anyone know where I can buy bottled time?

Paolo Amoroso, in noting the new, different screenshots of Climacs, a Common Lisp implementation of Emacs, says that “not yet ready for prime time”. However, since Paolo and I both reach an audience of Lisp-interested developers, let me emphasize what the casual reader might not have gleaned from that simple statement. It is fair to say that users reporting bugs will largely find their reports dropped on the floor, simply because the developers already know that the functionality isn't there or is already broken; on the other hand, anyone wishing to get their hands even just a little bit dirty will find that there is both a lot of low-hanging fruit and large potential rewards from hacking at the code.

In the meantime, Brian Mastenbrook is at the ILC to present a paper that we wrote (with Robert Strandh) about Climacs' current status and potential. I can't quite say that I regret not being there — perhaps I regret it not being here — but I do wish the “I” in ILC were taken a bit more seriously. (There are rumours to the effect that 2006 may see improvement in this area).

Mustard is surprisingly easy to make: take a random amount of mustard seeds, and grind until it feels too much like hard work. Add a small amount of vinegar (bonus points if it's tarragon vinegar, made by placing fresh tarragon leaves in a bottle of ordinary white wine vinegar) and approximately the same volume of water as mustard seeds. Grind some more, and leave to stand, tasting occasionally: the mixture will thicken and increase in sharpness over time. When it's eyewateringly strong, place in the fridge (or other cool place).

Armed with freshly-made mustard, it was then easy to make mustard mash and sausages in port reduction (non-permalink: 11th May entry); the reduction is made at the last minute, by boiling a splash of port with the cooking juices of the fried sausages.

Oh, yes, I've also fixed a rather difficult-to-diagnose bug in SBCL on Mac OS X:

  * bug fix: dynamically loading (via LOAD-SHARED-OBJECT or similar)
    "frameworks" on Mac OS X no longer causes an EXC_BAD_ACCESS if two
    or more runtime options were provided to the sbcl binary.

Thanks also to Brian Mastenbrook, Michael Hudson and Nikodemus Siivola.