Older blog entries for joolean (starting at number 64)


The ever-resourceful wingo just completed a series of patches that enhances the abilities of Guile's syntax expander. Among other wonders, the semantics of the `@@' form have been extended to support arbitrary expressions. To wit:

(@@ (my-module) (my-expr))

...will evaluate -- but, more importantly, expand -- (my-expr) in (my-module)'s environment.


I needed Andy's help with the above because, since last I wrote, I'd gotten pretty far with my implementation of the R6RS standard libraries for Guile. In fact, I'd pretty much finished with them -- all twenty-five or so of them (minus (rnrs bytevectors) and (rnrs io ports), which Ludovic Court├Ęs had already taken on), plus unit tests. And I got them (mostly) done in about one grueling month. (Granted, a lot of them are just repackagings of existing Guile functionality, but...)

Maybe this is how things always go. I spent about a year working on R6RS support in Guile: I was on the Acela back to New York City from the 2009 FSF Associate Members meeting when I started playing around with a naive form of the library-to-module transformer that wingo merged to `master' a couple of weeks ago; and spent the next eight months variously getting frustrated, learning `syntax-case', tweaking, and rewriting, before I got the code to the point where I could knock out all those library implementations in one shot.

And now they'll be in Guile 2.0, which is better than I'd hoped, even.


wingo just patched a pretty interesting issue I discovered in Guile's psyntax implementation. Guile uses a modified version of the expander that's module aware, so that lookups and bindings are done -- and hygiene is maintained -- with respect to module eval closures. Top-level definition forms, for example, such as `define' or `define-syntax', create bindings in whatever Guile thinks the "current" module is.

This works great, but there's a wrinkle added by the fact that the expansion process can change the current module. `define-module', for example, does this, by way of an `eval-when' -- after creating a new module, the modules system makes it current, so that if you've got a sequence of expressions like:

(define-module (foo))

(define foo 'foo) (define bar 'bar) (define baz 'baz)

...then `foo', `bar', and `baz' will be visible to `(foo)' and not to whatever module was current when you called `define-module'. The issue I found involved a slight modification to this pattern, wrapping everything in a `begin' form:

  (define-module (foo))
  (define-foo 'foo)
Semantically, this should be equivalent to the first form -- at the top level, `begin' splices its contents into the surrounding code as if it weren't even there, enabling you to produce multi-expression, uh, expressions, even in contexts where you're only allowed to produce one, like in the syntax transformer I was writing. But what I was seeing was that my definitions weren't creating bindings in the modules I'd placed them "within" -- particularly hard to troubleshoot when it came syntax definition, since the expander just allowed unbound custom syntax expressions to pass through to the evaluator, assuming they were procedure applications.

The root cause was that Guile didn't anticipate changes to the current module during the expansion of a single top-level form -- so the first form I described would be fine, since the expander would be re-initialized with the current module for each expression; but the second form would only check the current module once, for the `begin', and not after any of the expressions inside it. After Andy's patch, the part of the expander that handles top-level `begin' forms (`chi-top-sequence') checks the current module after each expression in the sequence to see if it's changed and updates the expansion environment appropriately.


All of the above has allowed me to make a fair bit of headway on the R6RS front. A while back I'd gotten started on implementations of the R6RS "standard libraries" and started merging them (along with test suites as appropriate) into Guile's module set (see the wip-r6rs-libraries branch), but I'd had to stop because of the difficulties caused by the issue above (which took me a very long time to isolate and properly articulate). Now I can start again!


...And another thing! This morning wingo pushed a couple of patches for the 1.9.x series that I'd been trying to get in for a while and which add two features that bring the capabilities of Guile's module system up to point where a "userland" (i.e., external to Guile's core) implementation of R6RS library support can be written. They are:

  • Support for the version information as part of Guile's "module" form, along with modifications to Guile's library search mechanism to support searching for modules matching an R6RS-compatible version reference
  • The ability to export a binding from a module under a different name
With these features in place, library support is much closer to being ready in time for 2.0.

I just pushed a patch for Guile that extends the unreleased 2.0 branch's Unicode support to include title case, as described in the Case Mappings section of the Unicode Standard. It's kind of complicated: In the context of characters, it's used with digraph characters (the canonical example being U+01F3 "dz") whose upcased form ("DZ") isn't appropriate for use at the beginning of word (where "Dz" would be a better fit).

What's interesting is that GNU libunistring, the Unicode library used by Guile defines the contract for uc_totitle such that it the function returns a special title case character, if one is defined for the specified character, otherwise it returns the upcased version of that character. In the context of strings, libunistring's title case mapping puts the first character of each word into title case as above and downcases all the other characters.

Guile has a set of predicates like char-lower-case?, which, under the hood, check for the presence of a specified character in a particular character set. In the original form of the patch, I had added a char-title-case? predicate which did the same for the title case character set. This led to situations in which

(char-title-case? (char-titlecase x))
would be false. We ended up taking it out.
wingo was in Brooklyn last weekend and we got to have a few pints at Double Windsor. We caught up on a bunch of stuff about the upcoming Guile 2.0 release this December -- the tower of compilers; swapping out Guile's legacy, synchronous garbage collector for libgc (which only just happened a week or so ago); and the possibility of porting Emacs' Lisp innards to Guile (which would really be a coup). I know I've said so already, but it's going to be totally sweet. Our meeting inspired me to get back on the R6RS library horse. So I'm working again on a series of patches to allow the use of version specifiers in Guile modules.

This involves some interesting searching / filtering operations, since the way we've opted to handle the wildcards and range specifications in R6RS's version reference syntax requires that the system choose the "best" match from a number of paths. It's not dissimilar to the search procedure used in CSS selector matching, although, thankfully, it's allowed to be a lot greedier.


Oh, man -- I think I might be the first Guiler on Advogato to announce this: Guile 1.9.0 is out! The 1.9.x series is unstable releases leading up to an eventual 2.0 later this year, and it's packed with enhancements and features that bear evaluation, especially if you've missed them in previous versions of Guile:

  • Guile now sports an actual virtual machine, meaning, among other things:
    • Scheme source can be compiled to bytecode for (much) faster loading and evaluation
    • Guile can finally compile and run code in languages other than Scheme! Initial support for ECMAScript 3.1 is included.
  • Robust multithreading via SRFI-18
  • Syntax-case macros are supported out of the box and maintain hygiene across module boundaries
  • Guile is now unicode-aware and has i18n support
  • Initial support for R6RS's I/O APIs

Grab the tarball here.

Wow, it's been a while.


After practically another year, SCSS 0.3.2 is out. I had to pretty much rewrite the parser from scratch, since the LALR implementation I was using (lalr-scm), while quite good, didn't give me as much flexibility as I needed with regard to error recovery. The new parser is based on the one in WebKit, and, while it's now pretty much 100% compliant with the CSS 2.1 recommendation, it's a bit slower than the LALR one. That's something I'll have to come back to, I guess. And I finally buckled down and added a test suite, at least for parsing -- testing by rendering pages in libRUIN's "ruinview" example program was a real time-sink.

Grab tarballs here.

R6RS Libraries

Another thing I sunk some time into over the past, well, year was trying to get a working R6RS-compatible libraries implementation into Guile. I've been frustrated for a while with the difficulties inherent in porting Scheme modules from one interpreter to another. For a while I thought Snow might be the solution to this, but after months of wrestling with its shortcomings and trying to get people interested in it, I concede that it's probably most useful as a distribution platform -- and R6RS's library specification, however onerous it might be to implement, is most likely to solve the platform-crossing problem. So I spent a while trying to map R6RS's requirements onto Guile's built-in module implementation, and I was almost successful, until some long-standing problems with Guile's management of hygiene across module boundaries reared their ugly heads. I asked the development list for help, and wingo, master of syntax that he is, stepped up and made some serious fixes, which are in the process of being tested and integrated. Guile 2.0, whenever it arrives, is going to have some pretty wonderful things in it.

I joined a LUG

...or, well, a Linux-oriented Meetup.com group. But that's still a big deal for me, as I'd ago given up on finding a serious Free Software presence in New York City (aside from NYLUG, which just... isn't my kind of vibe). At any rate, I found out about this one, The New York Linux Meetup Group while tabling at HOPE with mjording, who's kind of the organizer, I think -- and also a fellow Sunset Parker! They've got a monthly room reserved at 3rd Ward over in a (picturesquely) desolate part of Bushwick. Last month, the meeting included a pleasantly chaotic competitive attempt to develop the same toy web application in four different languages. This time around, I'm told it's just going to be a straight-up hackathon on personal projects, which suits me absolutely fine.


Tabling at The Last HOPE was a blast, kind of unexpectedly. I didn't go to any of the talks (although I got to see Jello Biafra's lengthy, meandering keynote on the big projector), but the FSF raised, well, I don't know if it's appropriate to discuss... but it was a lot. And we got a surprising number of requests for ladies' t-shirts.

Shout-outs are due to Matt, Ringo, and Thomas. ...And, grudgingly to Club-Mate. Club-Mate: One Gets Used To It.


After almost a year, SCSS 0.3.1 is out. This release features a full-scale code cleanup / reorganization, as well as major changes that improve portability and performance -- the lexer was rewritten by hand, and the pseudo-element handling no longer requires you to manager the style cache yourself. Huge!

Grab tarballs here


With encouragement from Ludovic, I spent nearly a month (of evenings and weekends) trying to write a more efficient pure-Scheme implementation of SRFI-89, and... failed. In the process, though, I learned a fair amount about Guile's handling of macros and lexical environments. I'm confident Guile will wind up with an SRFI-89 implementation in the near future; it'll just look pretty familiar.

I'll be tabling with the FSF at the HOPE conference tomorrow afternoon between 2:30 and 5:30 and all of Sunday afternoon. Drop by and say hi!

55 older entries...

New Advogato Features

New HTML Parser: The long-awaited libxml2 based HTML parser code is live. It needs further work but already handles most markup better than the original parser.

Keep up with the latest Advogato features by reading the Advogato status blog.

If you're a C programmer with some spare time, take a look at the mod_virgule project page and help us with one of the tasks on the ToDo list!