Older blog entries for lukeg (starting at number 33)

23 Oct 2004 (updated 23 Oct 2004 at 23:44 UTC) »

I went to the 10th annual Erlang conference on thursday and had a great time, as usual.

My favourite talk was about the Dialyzer (Discrepancy Analyzer). This program analyzes regular Erlang object-code to find discrepancies that suggest programming errors. The analysis seems very much like what CMUCL does: infer type information from the way variables are used, propagate it around, then warn about any contradictions. Very impressive that it runs on unmodified object code, appears to really find bugs, and has managed to consume a 1.1 million line Erlang program without exploding. These guys have done an exceptional job of designing their tool to actually appeal to working Erlang programmers. (So far it hasn't found as many bugs as I'd expected, but I'm not sure if that's good or bad.)

I also saw a really amazing demo of Virtutech's Simics product, which is only related to Erlang in that one of the HiPE guys has joined them. It's much like Bochs or QEMU but it emulates a wide range of machines: x86, AMD64, SunFire/sparc, Alpha, and so on. It has some extra-fancy features like the ability to run the machine in simulated time, so that when the kernel enters the idle-loop it "fast forwards" to the next event. Really amazing, but unfortunately it isn't free software and I gather it's priced only for large corporations.

And of course free beer at the post-conference ErLounge didn't go astray. Hurray for the Erlang companies who sponsored the event :-)


I'm currently implementing a protocol based on XML Schema and SOAP. As good as it was to spend five-odd years in blissful ignorance of XML, now I do wish that I'd started using it before XML-bashing became passé.

apenwarr: I think a really interesting case that's somewhat related to your Scripting Languages and Reliability question is Emacs Lisp. The standard Emacs distribution today includes about one million lines of Emacs Lisp, and though the language lacks basic features like a module system it's still comfortable to program with, even in such a large system. This amazes me.
20 Aug 2004 (updated 21 Aug 2004 at 00:01 UTC) »

I'm not luke@bluetail.com anymore, but I'm still @member.fsf.org.

ncm: Hi! I just noticed that you work at ITA Software. I have a question that you might be able to help me with, about this part of the article on paulgraham.com:

    9. We can do 10 seconds of Lisp computation on a 800mhz box and cons less than 5k of data. This is because we pre-allocate all data structures we need and die on queries that exceed them. This may make many Lisp programmers cringe, but with a 250 meg image and real-time constraints, we can't afford to generate garbage. For example, rather than using cons, we use "cons!", which grabs cells from an array of 10,000,000 cells we've preallocated and which gets reset every query.
I don't understand the CONS! bit. Doesn't this have the same performance characteristics as using a generation-scavenging collector with a 10MB nursery, forcing a GC between requests ("for free" because the nursery is pure garbage), and aborting if a GC is triggered before the request ends? Why don't you do it that way?
<tibo> Luke, you won't believe it, your photo is in a newspaper today, here in
<tibo> It is the picture that appears on your web page (assuming it is you).
<tibo> I'm serious !
<tibo> Well it not a very serious newspaper... The 10 lines "news" is about a
       "study" that concludes that 17% Germans adults "drink in order to
       become drunk"
<tibo> I will try to scan it.
The price of fame! :-)
Example 8
"C" provides a conditional expression.
Thus if "a" and "b" are integer vari-
        (a > b ? a : b)
is an expression whose value is that of
the larger of "a" and "b".
However this does not work if "a" and
"b" are to be regarded as unsigned 
integers. Hence there is a use for the
6326 max (a, b)
     char *a, *b;
       if (a > b)
The trick here is that "a" and "b",
having been declared as pointers to
characters are treated for comparison
purposes as unsigned integers.

I just saw in the paper that Christiania, the big hippie commune and soft-drugs haven in the middle of Copenhagen, was just raided by a hundred police. They put a lot of people in jail for a long time.

The world is less of a cool place now.

17 Aug 2003 (updated 17 Aug 2003 at 18:21 UTC) »
re: Clear and informative error messages

raph: CMU Common Lisp is a system that takes error-reportage with macro-expansion seriously.

In the case of compile-time errors in the results of macro expansion, CMUCL's error reportage gives some very useful information: the actual source expression that expands to erroneous code, some context to say which part of the expansion is in error, and an easy way to see how particular expressions get expanded (macroexpand-1) so that your eyeball can spot the error.

For errors occuring during macro-expansion, you get full source-level debugging, since the macro is a Lisp program.

Now I'm curious about how well it works for debugging runtime errors in macro-expanded code. I mustn't have done much of that.

Another more upbeat observation is how good programs like strace and ethereal are, and the fact that you can understand a lot about a program by looking at how it interacts with other programs. Is this what they call "stratified design"?

2 Aug 2003 (updated 2 Aug 2003 at 20:19 UTC) »

Today I wrote a small ethernet switch using my Lisp networking code. A ways back I broke it into separate "low-level networking library" and "partial TCP/IP stack" pieces, which makes it easier to write other applications like this switch. It's surprising how many useful throw-away applications have popped up at work recently when testing our latest network appliance.

I also added support for PF_PACKET sockets in addition to TAP interfaces. PF_PACKET sockets are better for switchey things because they directly access existing interfaces.

The switch is a toy, but I like it because it works and it fits on one screen. (Your screen mileage may vary.)

Here's the code:

;;; switch.lisp -- a toy ethernet switch built on `netlib'.
;;; (netlib source is at http://www.sourceforge.net/projects/slitch/)
(defpackage :switch
  (:use :common-lisp :netlib)
  (:export :start))
(in-package :switch)
(defvar *ports* nil
  "Array of switch ports (network devices).")
(defvar *fdb* (make-hash-table :test #'equalp)
  "Forwarding database: maps MAC address onto port number.")
(defun start (&rest devices)
  "Start switching packets between DEVICES."
  (setq *ports* (concatenate 'vector devices))
  (loop for device across *ports*
        for port from 0
        do (init-port device port)))
(defun init-port (device port)
  "Initialize DEVICE as an input port (number PORT)."
  (netdev-enable device
                 ;; This function is called when a frame arrives.
                 ;; FRAME is an ethernet frame as an array of bytes.
                 (lambda (frame) (input frame port))))
(defun input (frame input-port)
  "Process a FRAME arriving on INPUT-PORT."
  (multiple-value-bind (source destination) (header-addresses frame)
    (update-fdb source input-port)
    (let ((output-port (where-is destination)))
      (cond ((null output-port)
             (flood frame input-port))
            ((/= output-port input-port)
             (send frame output-port))))))
(defun header-addresses (frame)
  "Return the source and destination addresses from FRAME's ethernet header."
  (with-input-from-frame (stream frame)
    (let ((header (read-ethh stream)))
      (values (ethh-src header) (ethh-dest header)))))
(defun update-fdb (address port)
  "Update the forwarding database: ADDRESS is on PORT."
  (unless (ethernet-multicast-p address)
    (setf (gethash address *fdb*) port)))
(defun where-is (address)
  "Return the port that ADDRESS is on, or NIL if unknown."
  (gethash address *fdb*))
(defun send (frame output-port)
  (netdev-tx (aref *ports* output-port) frame))
(defun flood (frame input-port)
  "Send FRAME to all ports except INPUT-PORT."
  (dotimes (output-port (length *ports*))
    (unless (= output-port input-port)
      (send frame output-port))))

24 older entries...

New Advogato Features

New HTML Parser: The long-awaited libxml2 based HTML parser code is live. It needs further work but already handles most markup better than the original parser.

Keep up with the latest Advogato features by reading the Advogato status blog.

If you're a C programmer with some spare time, take a look at the mod_virgule project page and help us with one of the tasks on the ToDo list!