Fun with Lisp on Python
I’ve been hacking on a useless little project lately. My friend Brandon and I have been toying with the idea of designing a new programming language over the last couple of years. We are both compiler nerds, and many of our conversations usually lead to to a discussion about why this feature is great and why that language
sucks.
I’m a pretty big Python fan, and have been using it for years. It is still my go-to language when I just want to get something done. Unfortunately, the more I learn about programming languages like Common Lisp, Scheme, Haskell, etc., the more I run up against the restrictions that Python has.
Unfortunately, many of these are deliberate. It seams that Guido and many others resist features like continuations, macros, non-broken lambdas and proper tail recursion because they like to think of Python as a single-paradigm language. Adding these features means that some Python code could seem foreign or unfamiliar for some programmers.
I understand this argument, but on the other hand I find it extremely frustrating when I have an elegant, easy to understand solution to a problem that I just can’t use because of a seemingly arbitrary restriction.
So I thought I would kill two birds with one stone. I could start designing a language for fun (not profit :-)), and when I am ready to write the compiler, I can just emit Python bytecodes, so that I will have a huge set of libraries to experiment with, and have something usable right away.
This turned out to be extremely easy. Within a few days I had a working compiler that was near feature parity with Python, and some nice features added on. It is still pretty basic at the moment, I don’t have many of the features implemented that I have planned. For now it’s just a simple lisp-like language with similar semantics to
Python. After a couple of afternoons work, I did a google search for similar projects, and it turns out that there are about 10 people that have done something similar. I don’t really care though, this is just for fun.
One of the decisions that I made early on is that Python’s modules are a good idea. I think the module and namespace semantics are extremely well thought out in Python, and have encouraged a lot of really great patterns and idioms. So this language is definitely going to mirror that behavior.
One of the things I struggled with was under what circumstances should I allow myself to add syntactic sugar. I’m usually not a fan of such features, but there is one case where I made a compromise, and that is accessing attributes of an object. I tried to avoid this for as long as I could, but I tended to write a lot of code like this:
(set (getattr (getattr someobject 'propertya) 'propertyb) value)
In this case, I ended up settling for:
(set someobject:propertya:propertyb value)
I’m not really satisfied with that at the moment, but at least it is about the only such compromise I’ve made.
Another thing that is pretty important to me is avoiding any special operators. This is something that really bugs me about common lisp, I hate having to write code like this:
(reduce #'(lambda (x y) (* x y)) '(1 2 3))
Why can’t ‘*’ be a function? Scheme gets this right in most cases, but there are still around 10 special operators. I want zero special operators, or at least as few as possible. I ended up accomplishing this by allowing special operators to behave like functions in contexts where that made sense, and otherwise treating them like special operators. For example, the situations below would generate very different code:
(reduce * '(1 2 3))
This would simply map to a built in function for multiplication, that is only used when ‘*’ is used as a symbol. Otherwise:
(* 1 2)
This emits a normal BINARY_MULTIPLY instruction. This way it appears to have no special operators, but the code that is generated is as efficient as possible. I’ve tried to do this where ever I can.
At this point, it is possible to write just about any program you would write with Python, with a few minor exceptions that would be trivial to fix. Here is a simple pygtk program:
(import gtk)
(set my-window (gtk:Window))
(my-window:set_title "Hello, World!")
(my-window:connect "delete-event"
(lambda (w event)
(println "Goodbye, world!")
(gtk:main_quit)))
(my-window:show)
(gtk:main)
It really is remarkable how much Python resembles a Lisp internally. In some cases, what I thought would take days of hacking turned out to only take 1/2 hour. An example was loops. I wanted to implement something like the Common Lisp loop macro, only not as terrifying :-). It turns out that the behavior isn’t much different from Python list comprehensions, so implementing something like this was pretty trivial:
(loop for x in (range 10) for y in (range 10 20) if (not (= x y)) collect (x y))
Although in the end I simplified loop a great deal, it was fun to see it work with so little effort.
I don’t know if I will actually release this code. It isn’t really that useful, it’s just a fun little project to tinker with and an easy way to try out new experimental language features. But if you want to play with what I have now, you can get it from my Bazaar branch.
bzr branch http://codewalking.com/bzr/sketchy.dev
My plan is to continue experimenting with this, and eventually weed out all of the unnecessary features. At some point I imagine having something that would be worth rewriting, maybe targeting another VM with a JIT like the LLVM, the CLR or Java, or writing my own JIT (that would be fun!).
