Older blog entries for nikodemus (starting at number 49)

SBCL frozen

It's that time of the month again.

SBCL's frozen for testing, in preparation for 1.0.49 release due next weekend.

Some things on note this month:

  • Things like
    (defun foo (f &rest args)
      (apply f :bar t args))
    typically no longer cons the argument list on the heap.
  • Functions from files loaded as source have source location information associated with them.
  • Deadlocks on mutexes and spinlocks are detected.
  • --script commandline option now also works with standard input, and is generally more shell-scripting friendly.
  • Using call-next-method with explicit arguments in safe code is much faster in the common case of required arguments not changing. Eg:
    (defmethod foo :around (x &key bar)
      (call-next-method x :quux t :bar (compute-default x bar)))
  • Few instances where interrupts were disabled needlessly have been fixed, leading to easier to debug code.

That's not the complete list by any means, which is why you should grab the latest from git, eyeball the NEWS, and give it a whirl -- and report any regressions.

Syndicated 2011-05-30 19:18:30 from Nikodemus Siivola

Improvements to slime-indentation

I just merged a mess of improvements to the slime-indentation contrib. To gain the benefit of them, update your Slime from CVS and make sure your slime-setup call looks something like this:

(slime-setup '(slime-fancy slime-indentation))
;; This is my preference -- alter to taste.
(setq lisp-lambda-list-keyword-parameter-alignment t)
(setq lisp-lambda-list-keyword-alignment t)

Summary of improvements:

  • Subclause aware loop indentation. (Adapted from cl-indent-patches.el.)
  • Support for if* indentation. (Adapted from Gabor Melis' mail to slime-devel.)
  • Improved lambda-list indentation. (Didier Verna's recent work.)
  • Improved defmethod indentation. (Didier Verna's recent work.)
  • Support for #+foo and friends.
  • Indent ,(...) and ,@(...)as code.
  • Correct handling of (... &rest symbol) indentation specs. Fixes eg. prog indentation.
  • Distinguish default from other things beginning with "def". Fixes eg. :default-initargs indentation.

Syndicated 2011-05-15 17:32:55 from Nikodemus Siivola

SBCL 1.0.48 released

It's been a while since I last summarized SBCL status, so here's my five favorite changes in the 1.0.43-1.0.48 range (out of over 100 user-visible ones).

  • Better toplevel form reporting from eval, ie. no more "in: LAMBDA NIL" in compiler notes.
  • Faster slot-value in the presence of slot-value-using-class.
  • Optimized constructors can be used for make-instance in the presence of slot-value-using-class and initialize-instance :around methods.
  • Support for ~/ and ~user/ in pathnames.
  • A number of improvements to compile-time typechecking and derivation, eg. full warnings for violating a proclaimed ftype.

Additionally, if you're feeling the least bit adventurous, the current devhead adds source-locations for code loaded as source -- with it M-. can take you to functions defined in your initialization files, etc.

Syndicated 2011-05-10 06:33:02 from Nikodemus Siivola

There's Indentation In The Air

Didier Verna's recent efforts in lambda-list indentation motivated me to look into two of my pet irritants.

  • Forms beginning with default indented as if they were defun forms.
  • Broken prog indentation.

After poking around sufficiently to fix them locally, I started thinking about how to make improvements like this as painless as possible -- because let's face it, there are still other issues left. cl-indent.el is distributed with Emacs/XEmacs, which makes it a somewhat slow moving target.

My plan: (1) copy current cl-indent.el to Slime as contrib/slime-cl-indent.el, replacing the ancient version already included as part of slime-indentation contrib. (2) improve it without overly worrying about pushing all changes immediately to Emacs propers.

While waiting for input from other Slime folks, I have a tree that does just that at GitHub: https://github.com/nikodemus/Slime/tree/slime-indentation. It incorporates both Didier's stuff and my more modest changes.

Do you have cl-indent.el fixes or improvements stashed away somewhere? Fork that tree, merge your stuff, and poke at me to merge it back. ...then we'll see about getting it to Slime proper.

Syndicated 2011-05-08 13:34:08 from Nikodemus Siivola

Slow Fontification That Isn't, and Human Nature

Yesterday I mentioned Slime and font-lock-verbose.

Turns out I didn't have my story quite right. It isn't fontification that's slow, but displaying the progressing bar of dots. ...which makes the whole thing even sillier.

Now I might have twigged on to this had I stopped to think about why this irritates me on OS X, but not on Linux -- the issue is much reduced on the X-server backend, and nonexistent on a terminal Emacs.

It's hard to be rational when you're trying to focus on something and an unrelated issue keeps getting in your way. The last thing you want to think about is the unrelated annoyance.

So, I promise to be patient the next time someone reports a bug and doesn't seem to have spent of single thought on the issue -- and hence neglects to mention the tidbits that would have allowed me to diagnose the issue. At least they reported it.

Syndicated 2011-04-25 08:36:29 from Nikodemus Siivola

Public Service Announcement: set font-lock-verbose to nil

The time it takes Emacs to fontify *slime-compilation* every time I hit C-c C-c has been irritating me for ages. Enough so that I often opt for alternatives such as C-x C-e (which doesn't provide source-location information for later.)

There is a simple fix.

;;; In your .emacs
(setq font-lock-verbose nil)

You don't lose anything, except having to wait while Emacs fontifies the damn thing -- which doesn't even pop up for C-c C-c.

Smart folks must have known this before, but I'm not that smart. If you were one of us unfortunates, put that in your .emacs and feel the quality of your life improve.

Much kudos for Tobias Rittweiler for the tip.

Syndicated 2011-04-24 10:45:40 from Nikodemus Siivola

Learning the Temper of Steel

I've started a separate training diary so that I don't have to wonder about crossing the streams--from now on this blog will only contain computery stuff, promise.

(While things like Planet Lisp can easily filter out entries like this based on tags, I suspect asking the 0.5 persons who might want to read my training diary to filter out the rest is a bit much...)

Syndicated 2011-03-05 09:56:22 from Nikodemus Siivola

Optimizing Lookup Functions Using LOAD-TIME-VALUE

Consider code along the lines of this:

(defvar *foo-table* (make-hash-table))

(defun find-foo (name &optional errorp)
  (or (gethash name *foo-table*)
      (when errorp
        (error "No FOO called ~S." name))))

;;; Style Cop says: It is good form to keep the interfaces identical,
;;; even though the SETF version doesn't use the ERRORP.
(defun (setf find-foo) (foo name &optional errorp)
  (declare (ignore errorp))
  (setf (gethash name *foo-table*) foo))

Assuming that cases with constant NAME arguments exist, how to optimize them -- aside from custom hashing schemes, etc?

  1. Make *FOO-TABLE* hold cells holding the actual objects.
  2. Use LOAD-TIME-VALUE to grab hold of the cell inline.
  3. (SETF FIND-FOO) will first grab the cell and then update it.


(defvar *foo-table* (make-hash-table))

(defun find-foo-cell (name create)
  (or (gethash name *foo-table*)
      (when create
        (setf (gethash name *foo-table*)
              (cons name nil)))))

(defun foo-from-cell (cell errorp &optional name)
  (or (cdr cell)
      (when errorp
        (error "No FOO called ~S." (or (car cell) name)))))

(defun find-foo (name &optional errorp)
  (foo-from-cell (find-foo-cell name nil) errorp name))

(define-compiler-macro find-foo (&whole form name &optional errorp)
  (if (constantp name)
      `(foo-from-cell (load-time-value (find-foo-cell ,name t)) ,errorp)

(defun (setf find-foo) (foo name &optional errorp)
  (declare (ignore errorp))
  (setf (cdr (find-foo-cell name t)) foo))

(define-compiler-macro (setf find-foo)
    (&whole form value name &optional errorp)
  (declare (ignore errorp))
  (if (constantp name)
      `(setf (cdr (load-time-value (find-foo-cell ,name t))) ,value)

...and then there are no hash-table accesses at runtime for the constant argument cases.

Depending on your implementation's support for SETF-compiler-macros, you may need to replace the SETF-function with>

(defsetf find-foo set-foo) ; then SET-FOO and a compiler-macro for it

... but the same principle holds.

Syndicated 2011-03-01 13:34:04 from Nikodemus Siivola

Fixed-Width Fonts by Force

A quick break SBCL hackery and lisp pontification.

When I use Gmail through the web-interface, I want fixed-width fonts since most of my messages deal with code. I also want a fixed-width font in textareas, not just on Gmail, but pretty much always.

The following bits have been in my userContent.css for quite a while now--and have survived a number of Gmail updates. So, in the hopes that another fixed-width grognard like me will find this useful, I offer the following (adjust font-size to taste):

/* Fixed-width textareas everywhere. */
textarea {
    font-family: MonoSpace !important;
    font-size: 14px !important;

@-moz-document domain(mail.google.com)
    /* GMail messages. */
    div.ii, input {
        font-family: MonoSpace !important;
	font-size: 14px !important;

Syndicated 2011-02-27 09:32:08 from Nikodemus Siivola

Inline Caches with LOAD-TIME-VALUE, part 2

So why would you want to use LOAD-TIME-VALUE to implement a cache instead of doing something like this:

(let (cache)
  (defun parse-timestamp (string)
    (let ((cached cache))
      (if (equal string (car cached))
          (cdr cached)
          (let ((stamp (%parse-timestamp string)))
            (setf cache (cons string stamp))

...or this?

(defparameter *cache* nil)

(defun parse-timestamp (string)
  (let ((cached *cache*))
    (if (equal (car cached) string)
        (cdr cached)
        (let ((stamp (%parse-timestamp string)))
          (setf *cache* (cons string stamp))

It's the reason why I'm talking about inline caches. While using toplevel closure or a global variable to implement the cache works fine, consider inlining (and compiler-macros.)

Simply by proclaiming the LOAD-TIME-VALUE-using function INLINE, you will be giving each call site their own cache--which is often exactly what you want. (Admittedly this timestamp parsing probably doesn't warrant that.)

Similarly, you can define the a two-level cache with both local/call-site and global lookups (though again, timestamp parsing almost certainly doesn't need this):

(defvar *global-timestamp-cache* nil)
(defconstant +timestamp-cache-max-size+ 12)

;;; A global lookup with a simple size-limited cache.
(defun parse-timestamp (string)
  (let* ((cache *global-timestamp-cache*)
         (stamp (cdr (assoc string cache :test #'equal))))
    (or stamp
        (let ((stamp (%%parse-timestamp string)))
          (setf *global-timestamp-cache*
                (cons (cons string stamp)
                      (if (>= (length cache) +timestamp-cache-max-size+)
                          (subseq cache 0 (1- +timestamp-cache-max-size+))

;;; Two cases: constant string arguments get constant timestamps,
;;; and more importantly, we add a per-call-site 1-element cache.
(define-compiler-macro parse-timestamp (string)
  (if (stringp string)
      `',(%parse-timestamp string)
      (alexandria:with-gensyms (cache cached stamp)
        (alexandria:once-only (string)
          `(let* ((,cache (load-time-value (cons 'local-timestamp-cache nil)))
                 (,cached (cdr ,cache)))
            ;; NOTINLINE inhibits compiler-macro expansion.
            (declare (notinline parse-timestamp))
            (if (equal ,string (car ,cached))
                (cdr ,cached)
                (let ((,stamp (parse-timestamp ,string)))
                  (setf (cdr ,cache) (cons ,string ,stamp))

I keep saying that timestamp parsing doesn't need local (call-site) caches. What does?

  1. Things that are in the really critical path where avoiding a function call is a win.
  2. Things where there are multiple arguments ("keys"), and one or more of them are often constant: (foo 'bar 'quux zot). In this case the local lookups are much more likely to result in a cache hit because only one argument out of three varies--and since the local lookup only needs to index using a single key, it is always going to be faster than the global cache lookup which needs to index using all three keys/arguments.

Final note regarding caches, and inline caches in particular: don't go overboard with them. Unless you actually need them, they will hinder your performance more than they help it: a cache miss means extra work done compared to a straight up call, so unless the number of cache hits is sufficient to compensate for that they're not worth it. As always, the profiler is your friend.

Syndicated 2011-02-21 06:25:42 from Nikodemus Siivola

40 older entries...

New Advogato Features

New HTML Parser: The long-awaited libxml2 based HTML parser code is live. It needs further work but already handles most markup better than the original parser.

Keep up with the latest Advogato features by reading the Advogato status blog.

If you're a C programmer with some spare time, take a look at the mod_virgule project page and help us with one of the tasks on the ToDo list!