Older blog entries for company (starting at number 20)

Byzanz 0.1.0

I've now arrived at what I'd call feature complete. It even has an about box! So, go grab Byzanz 0.1.0 and play with it.

So what's new in this release? Three things: The applet got a complete overhaul, you can now record your mouse pointer and the code has been moved to Gnome CVS. What's also new is me wondering about where I want to go with this thing. Should it be included in the Gnome desktop? Should it be made to record to other formats? If so, which? Do people want to record audio with it? Do people want to add things to these recordings (like bubbles explaining things) and need an editor for this? Do people even care? I guess I'll need to wait for recordings popping up on the net to see what people use it for.

Another thing I've played around with a lot while writing this thing is GIF encoding and decoding. Encoding is interesting for two reasons. The first is performance. You want to ideally have your GIF completely encoded when you press the stop button and not have to wait until it's done encoding. While for an average desktop session, this is no problem even on my old iBook, recording playback of a full screen H264 movie (which itself can take more than half of the performance of a 3GHz CPU) will have you wait a while for the result. So what's so slow in the encoder?

• Color lookup is slow. Color lookup is the process of selecting that one of the 256 colors in a GIF that most closely matches the color we want to encode. I've yet to find an algorithm that implements that fast and efficient. The current tree-based algorithm and the "create a 64MB lookup table" approach seem both suboptimal to me - the first is slow, the second uses far too much memory.
• Dithering is slow. There's a lot of calculations required per pixel to keep track of all the different error values. This could possibly be sped up quite significiantly by using vectorizations (Altivec, MMX, SSE), but I'm not a specialist on those.

The second thing that is interesting about encoding is the size of the encoded image. The current code already optimizes quite a bit. It tries to find regions that haven't changed and omits them or if that's not possible it sets them as transparent. That way you get a lot of transparent pixels, which result in a pixel array that can be compressed a lot better. You can see this by opening one of your recorded GIFs in the GIMP (which is an excellent debugging tool for animated GIFs btw). But there are of course still optimizations that might be useful, like marking a pixel transparent even though it changed, when it didn't change a lot.

Another interesting thing are GIF decoders. Since your average GIF image is small (you know those little animations on the web - the biggest ones are probably banner ads), but Byzanz easily records full screen animations that last multiple minutes and easily exceed 50MB in size, you find the limitations of the current GIF decoding libraries easily. And you find out that every project has its own GIF decoder. And you find out when you record a 17MB animation of a short movie with around 1500 frames (the movie itself is 9MB in size, so don't use GIFs to record movies ;)), that

• the GIMP decodes the whole image into layers and then gets performance issues when rendering the resulting file, since it has to compose 1500 layers. Changing the visibility of one layer result in a huge lag. It's ok in memory usage though (around 300MB, probably only keeping the change regions).
• Mozilla also decodes the whole image at once resulting in rendering hickups during playback of the loading animation. When it plays back the looping animation a second time, everything works fine though. It uses as much memory as the GIMP for this (300MB).
• Internet Explorer 4 (the only one I have) opens the image fine and displays it without problems. No clue about memory usage though.
• Gtk consumes all available memory and my machine hangs. (Crappy OOM handling therefor a desktop, Linux.) This is because it tries to keep the completely rendered frames in memory. That easily exceeds multiple Gigabytes.
• note to self: test Konqueror

None of those applications handle GIF the way it was envisioned in the GIF spec: As a data stream. They all implement it like it's used most of the time: As ad banners. I wonder if Gtk and Mozilla devs would like to get their gif image handling code improved, even if that'd require writing a new library. I'd bet on "don't care really, the current code is good enough".

Oh and one thing: I need icons for Byzanz that don't suck. Feel free to create some and send them my way - or just check them in if you have a Gnome CVS account.

Byzanz 0.0.3

Byzanz 0.0.3 is out. This time it's supposed to work on a lot more computers than jus my laptops. The last release had the (expected) first release illnesses and for most machines didn't work at all. Particularly the "everything is red" and "Byzanz needs XComposite enabled" bugs are gone. A new demo of Gedit's new features is available straight from Paolo's blog

I was really overwhelmed by the feedback I got and how many people expressed interest. Such things really motivate. Thanks to everyone who emailed me with bug reports, tips or a translation. It's greatly appreciated. If this release actually works for normal people, I'm particularly interested in if it works on 64bit systems.

I should probably mention one thing: The whole code does not do error checking. So if something fails, it fails and it won't tell you or do something against it. Error reporting has to wait for one of the next releases I'm afraid. But then again, this is 0.0.2. ;)

Writing panel applets

Writing good panel applets is hard. And since pictures say more than a thousand words, look at this:

This is two very standard widgets given less size than requested. And I'll explain my big hurdles to writing good panel applets anyway:

• Gtk widgets are really bad at handling the case of getting allocated less space than they requested (see the screenshots above). This is because the Gtk usecase is a window which sets its minimum size to the requested size of its contents, so it basically never happens. But for a panel this is a frequent happening, because the user sets the maximum size of the panel and not Gtk.
• There are no containers that handle the different panels in a satisfying way. Since it is possible to have panels on all 4 sides on the screen (vertical and horizontal orientation) and with sizes from 23 to 120 pixels, there is a lot that can be and is done wrong with widget layout. The simplest example is that a container that behaves like a Vbox on a vertical panel should behave like a HBox on a horizontal panel.
• Panel applets must be optimized for size. Since they occupy parts of your screen all the time, the space occupied should be as small as possible. A lot of widgets are written for nice display in big windows (as in 500x300 pixels or more) and don't care about using 2 or 3 pixels more. A GtkButton in the most-used themes uses on average 5 pixels on both sides for displaying the box around it. If you now remember that a normal panel is 30 pixels high, you can imagine that losing a third of your available space just for a nice border is not what you want.
• Since the presentation space of applets is not very big, the information must be presented in a very compact form. For example text is not an option most of the time, since text is just too big. ("Hello World" requires 76x16 pixels in my configuration, that's almost enough to display 3 24x24 images.) Unfortunately there seem to not many widgets around that try to achieve the same goal. An exception might be the GtkToolbar widgets, but then again the toolbar uses a visually very different approach from panel applets, so just using toolbar items would look kinda stupid in a panel.
• Applets normally don't react to right clicks but bring up the default menu instead. But lots of Gtk widgets intercept right clicks. This makes applets with those widgets feel wrong. (The window list does this for example)

So what I'd like to have is a collection of widgets that make it easy to create good applet UIs. And I'd like
And I'd like these widgets applied to the current applets. Current applets suffer heavily from this not being available. Just try using a 60px vertical panel as your only panel for a while to figure this out ;)
Unfortunately there seems to not be a lot of interest in getting applets working nicely the maintainers of that code tell me. I wonder why that is. Probably just a case of missing good libraries to use, as always...

Edit: Yes, that read 0.0.2 before, but people seem to have their own partition for $HOME which tends to cause issues. So I did a quick-fix to 0.0.3 for them.

Byzanz

It took me a while, but I'm now able to do useful things with it. Or said differently: Byzanz 0.0.1 is ready for everyone brave enough to play with it.

What is Byzanz?

Byzanz is a desktop recorder. Just like Istanbul. But it doesn't record to Ogg Theora, but to GIF.

Screenshots?

Here's one you can make with it:

The idea for that demo was taken from Richard's blog. He said he needed quite a while in Gimp for his pictures. I needed around 30 seconds. That link goes to a 1.6MB recording of the session I used to record the above demo.

Why GIF?

GIF is the best format I found for recording desktop sessions. The criteria I wanted were these:

• You see use it in any browser. Preferrably without plugins.
• It records a typical interesting desktop losslessly or at least in an acceptable quality and in an ok filesize.
• It is free.
• Recording works fast enough.

What's a "typical interesting desktop?"

• Short tech demos like the one from Richard or similar.
• Bigger tutorial style demos that are currently typically done in Shockwave/Flash. And I can't see those.
• "Do this to reproduce the bug"
• And my original reason was: I wanted to demo my ideas for animated themes like this one:
  

Why the name?

I couldn't come up with a better one and thought it's a clever pun on someone else and his application. ;)
And it's a name that's not yet taken. At least apt-file search byzanz doesn't return anything.

Update: There's been quite a few people testing it (cool!) and writing me mails about one thing I forgot to mention: Byzanz currently requires a running Composite manager on your X server to work. If you think you're brave enough to try this, see here for a good tutorial on how to make it work. Otherwise wait for me to fix it in a future release.

Havoc has a big rant about why the iPod rules and all the other players don't. This rant is probably correct for the US, but it fails for Europe. Because all his arguments are based on the fact that the iPod is the dominant music player. And in Europe, it isn't. When I'm having my daily commute in the subway, you can see some people with iPods, but most of the people use so called mp3 sticks. And of course I have reasons for why I believe the iPod fails in Europe:

• Because these sticks are dirt cheap (you can get them for less than $50 and even when they were you they cost less than $100 pretty fast). They are in fact so cheap, it's a convenience product. The iPod is expesive and definitely not convenience.
• Music players aren't cool (at least not with people I talk to). Mobile phones are. And conceerning the youth ringtones are definitely cooler than a whole song. Ican only imagine what an Apple mobile phone would result in...
• Apple doesn't really live up to the "all your music" claim. People here use Windows and Winamp or Windows Media Player. So using the iPod would either mean switching your music playback program or copying your songs manually in Windows Explorer. All your music?
• I don't know a single person that had even considered buying anything at iTMS (or at any of the other music shops that do far more advertising around here, like Musicload). People I know trade CDs and burn them and that doesn't work with iTMS purchased songs.

So I guess what it all comes down to is that I think that the iPod isn't doing so well because it's so well designed, but because Apple managed to make it synonym with "mp3 player" and especially "cool mp3 player". But it only managed that in the US.

When NASA was trying to go to the moon, there was a great deal of enthusiasm: it was a goal everyone was anxious to achieve. They didn't know if theys could do it, but they were all working together.

I have this idea because I worked at Los Alamos, and I experienced the tension and the pressure of everybody working together to make the atomic bomb. When somebody's having a problem - say, with the detonator - everybody knows that it's a big problem, they're thinking of ways to beat it, they're making suggestions, and when they hear about the solution they're excited, because that means their work is now useful: if the detonator didn't work, the bomb wouldn't work.

I figured the same thing had gone on at NASA in the early days: if the space suit didn't work, they couldn't go to the moon. So everybody's interested in everybody else's problems.

But then, when the moon project was over, NASA had all these people together: there's a big organization in Houston and a big organization in Huntsville, not to mention at Kennedy, in Florida. You don't want to fire people and send them out in the street when you're done with a big project, so the problem is, what to do?

Richard P. Feynman, What do you care what other people think?, p.214f., WW Norton & Company 1988, ISBN 0-393-02659-0

s/NASA/Gnome/
s/moon project/Gnome 2.0/
s/Houston/Utah/
...

I'll leave it as an excercise to the reader to figure out if NASA has gone uphill or downhill since reaching the moon.
The whole book this quote is from was an interesting read for me, especially with the parrallels between how NASA and Gnome evolve. And this quote sums it up best.

Says Rodrigo: These notifications expire a few seconds after they are shown, so they should not disturb the user at all.

The first thing that came to mind when I read this was that this line is the exact justification for popup advertisements in browsers. Some more advanced popups using DIV even expire themselves, so they meet pretty much the exact same criteria as notification popus.

I'm still surprised why so many people, especially those which do a lot of usability work in Gnome think it's a good idea to use popups. Popups do two things that have been identified as very bad in a lot of other contexts:

1. They occupy screen estate not assigned to them. To keep his screenshots as an example: The bottom right is where a lot of status bar icons in applications are.
2. They are animated. They're not containing animations, but the popping up and fading out of the notification is an animation. Unexpected animations cause interrupts in workflow. OTOH, Clippy is really cute, and you can switch him to look like Einstein.
3. Another thing not related to popups but related to libnotify is discoverability. If I'm away from my computer for 10 seconds I might have missed a notification.

So the question is how notifications should be handled? Well, there's different kinds of notifications:

1. Notifications the user wants to be interrupted for. Such notifications would be allowed and even should try to interrupt the user. Examples here are low battery on laptops, full hard drive, but also me waiting for a download to finish or a friend to come online and just browsing the web until that happens.
2. Notifications the user wants to look at later. This is mostly stuff where the user decided that things he's interested in haven't happened yet and then diverts his attention elsewhere, but will come back later to check. Examples here are downloading a movie someone linked to, checking email or friends going on/offline in IM.
3. Notifications that are uninteresting and the user will probably never want to look at. Examples here are all the join/part messages on IRC or IM, the temperature outside changing or the clock ticking.

I think these 3 categories pretty much sum up the different uses for notifications. At least I've not found any important steps missing. But feel free to enlighten me. An important thing to notice is that for different users different messages are important.

I think I'll now just sum up some ways that I have found for notification handling and how I feel about them.

• Windows XP pops up bubbles all over the place telling you things like "Where are my icons gone?" or "Where are my programs?", especially after installation to get people aware of the new features compared to older Windows versions. After I have installed XP for the fifth time, this is just annoying.
• Firefox (at least on Windows) pops up a little box in the top right to inform that all downloads are complete. Apart from the fact that it doesn't work correctly with fullscreen applications (read: games), it mostly fails to notify me of what I'm interested in: Most of the time it's the first download to complete, not the last. And there's the issue of discoverability. If I've been looking away for a short while, the box has already disappeared, so I might not notice the download having finished.
• All browsers (and also IRC and chat programs) use Icons or colors in the tabs to tell you when a tab has finished loading. This is a really nice notification, because it uses a space pre-assigned for this task and is easily discoverable. (Imagine libnotify would popup a little box for every page that finished loading... ;))
• Applications on Windows are allowed to flash their taskbar entry to notify. This is a nice feature, because it only occupies space assigned to them. Slightly bad things about this from my point of view is that it's constantly blinking, which tends to annoy me so I click the app to turn it off - I'd prefer if it'd just change the color of the taskbar and not blink and there's sometimes and issue with discoverability if I'm using an auto-hiding panel.
• Amarok uses an OSD for song change notification. While a funny little gimmick, it really gets annoying when something pops up on your screen every 3 minutes.
• Lots of applications have voice messages to signal notifications. Those are quite nice, unless I'm listening to something important (like when coding audio applications), don't want to be interrupted (like when coding), when I'm not at my desk (discoverability again) or when they happen far to often (Teamspeak's "player joined" message really is that annoying). In the end they're a great addition, but not a good mechanism in itself.
• Warcraft 3, Unreal Tournament and I suppose lots of other games too print notification texts pretty much centered and in a big font onto the screen. This works very well even though it's essentially a popup. The reasons for this I think are: 1) The message doesn't really occupy a lot of screen since the background is transparent and I only see the text message (slowly fading out), so I can easily see what's happening behind it. 2) The message is important, so I want to read it anyway. 3) There's no discoverability problem, because while playing I'm not away. 4) I don't perceive it as an animation while it's fading out, so I'll look at it when it pops up and can ignore it after that.
• Warcraft 3 offers a machanism to jump to the current notification. In Warcraft 3 this is a huge feature since it allows centering the view at where the action's at. Especially since the key is unused otherwise. The key is space btw.

So, summing it up, I think I like notifications that don't occupy screen estate they weren't supposed to use, don't divert my attention when they're not important to me and are easily discoverable even when I've not been there for a while. I have no clue how to ideally do that, but I know that the applet popups from Rodrigo fail on all 3 of these points.

The fun debate

I think the real reasons why it's not fun to hack on Gnome have nothing to do with the usefulness of the API, libs, platform or whatever. After all, Linus sounds like it was fun to write a kernel from scratch. There's so much interesting stuff to hack on (at least for me), there's no technical reason why Gnome hacking shouldn't be fun for at least the next ten years. No, I guess the real reason - and this has been touched by Seth , Martin and James without really noticing it: It's the community. It's how you feel when you communicate with the people the make up the project you hack on. And let me tell you one thing: The Gnome and GStreamer communities were great fun to be in, but they're less and less so.

I always compare the Gnome community some years ago to a college community, while the current community to me looks more like a corporate community. The names are chosen to show what the primary values for this communities are. You'll probably come up with better definitions than I can, so I'll just give some examples:

• Some time ago there wasSullivan. This was the last such big (typical college level) joke I'm aware of in Gnome. The file in that link was last modified one and a half years ago.
• I'm sure D-Bus is a cool technology, it's just that noone seems to care about it. At least there's noone on IRC that I could ask when I have a question. Yes, there's a mailing list, but mailing lists are not a good idea when you just have a small question. They are very bad when you have followup questions and they outright suck if you want to have a discussion.
• In fact IRC vs mailing lists is a good example of college vs community culture. IRC is a fast, informal and lossy (in the sense that you sometimes have to ask stuff twice) way to communicate with each other. Sounds a lot like talking at parties. Mailing lists instead are a slower, formal, archived and regulated way of communicating. Sounds a lot like filing taxes to me. Both are definitely necessary, but you decide what's more fun.
• A lot of discussion about directions, design and whatnot happens out of reach of the broader community. I'm currently witnessing this in the GStreamer project, where the next version is practically designed behind "closed doors". These closed doors are more often than not the office space of a company (in the GStreamer case it's really nice offices btw). But when half of the community is part of a company and somehow locks me out, there's not a lot of fun in it.
• If the most important thing about a patch is to be sure to not break ABI compatibility, it's certainly less fun than when the most important thing about a patch is what cool new features it adds. (I am fully aware that compatibility is important and I don't think that should change, but it definitely is less fun).
• The conference that represets the community costs money. Nuff said.
• The treatment of newcomers is interesting, too. College communites tend to welcome people with open arms. On IRC they are just integrated into the running discussion. Corporate communities tend to not have IRC channels and answer with one-liners on mailing lists. (See the D-Bus part above).
• Fun communities have fans.

When I first joined Gnome, it was fun and I thought "Hey, I'd like to do that for a living." These days with people even quitting their jobs working on Gnome, I'm not so sure.

What I can do and you can't

I put this in /usr/lib/gstreamer-0.8/python, run gst-register and have a new element available that does what it says.

import gst
import gobject

 class ChannelSwap (gst.Element):
  __gsttemplates__ = (
      gst.PadTemplate ("src", gst.PAD_SRC, gst.PAD_ALWAYS,
        gst.Caps ("audio/x-raw-int, width=16, depth=16, channels=2")),
      gst.PadTemplate ("sink", gst.PAD_SINK, gst.PAD_ALWAYS,
        gst.Caps ("audio/x-raw-int, width=16, depth=16, channels=2"))
  )
  __gstdetails__ = ("Channel swapper", "Audio/Filter", "swap left and right channel",
      "Benjamin Otte <otte@gnome.org>")

   def __init__ (self):
    self.__gobject_init__()
    self.srcpad = gst.Pad (self.__gsttemplates__[0])
    self.srcpad.set_link_function (self.link)
    self.srcpad.set_getcaps_function (self.getcaps)
    self.add_pad (self.srcpad)
    self.sinkpad = gst.Pad (self.__gsttemplates__[1])
    self.sinkpad.set_chain_function (self.chain)
    self.sinkpad.set_link_function (self.link)
    self.sinkpad.set_getcaps_function (self.getcaps)
    self.add_pad (self.sinkpad)

   def otherpad (self, pad):
    if self.sinkpad == pad:
      return self.srcpad
    else:
      return self.sinkpad

   def link (self, pad, caps):
    return self.otherpad(pad).try_set_caps (caps)

   def getcaps (self, pad):
    return self.otherpad(pad).get_allowed_caps ()

   def chain (self, pad, data):
    data = data.copy_on_write ()
    for i in range (0, len (data), 4):
      tmp = data[i:i+2]
      data[i:i+2] = data[i+2:i+4]
      data[i+2:i+4] = tmp
    self.srcpad.push (data)

 gobject.type_register (ChannelSwap)
gst.element_register (ChannelSwap, "channelswap")

And no, it's not performant. Now to write the mail with the (quite big) patch to the list, so you can do that, too. You can thank jdahlin for making me start hacking on Python btw.

Wanna puzzle?

Since rbultje promised me a coke, I present this to him and the rest of the world. (Screenshot! Go click the link!)
I'm convinced now that puzzling moving images is far harder than the standard image. If you like to play, the code is in gst-plugins CVS. If you also got a video player that forwards key presses (which buggy totem doesn't do here), you can just change your videosink to "puzzle ! xvimagesink" and start puzzling. If you're too puzzled, hitting space restores the normal picture. :)

If you're really hardcore, you watch a movie like The Matrix with lots of cuts and set your videosink to "puzzle rows=30 columns=40 ! xvimagesink" to get a puzzle with 1200 pieces. Maybe you'll even have solved it before the movie ends ;)

(updated, I was kind of in a hurry when I started writing it. And yes, I missed my bus.)

Since Colin and Havoc are into version control systems, I'll offer my thoughts, too. This is mainly what I realized when doing some of my code in arch (using tla) and some of it in CVS. It's also mostly evolutionary and doesn't try to go as far as Havoc's ideas.

• Please honour the structure of my project and allow me to use it. In CVS I can commit, diff, update and whatnot from any directory, with tla I need to use the top source directory.
• Don't clutter my working directory or invent other requirements. It is my working directory and I intend to work with it. This includes searching through it (tla puts lots of changesets or whatever into the .{arch} dir and these end up being duplicates in my searches). It includes moving the whole working directory elsewhere and continuing to work with it (tla caches inodes, so all hell breaks lose when I copy it to my laptop and try to do stuff from there). And it includes moving around files to better organize my source tree (not possible in CVS unless you move the files on the server).
• Allow changing all user-readable identifiers easily. I named one of my first repos for testing arch "company@localhost". When I later wanted to publish it, I couldn't just rename it to something sensible like "otte@gnome.org". Now everyone can happily use "company@localhost" for a while.
• Give me an easy way to work with a file's history. I'm currently doing this via ViewCVS mostly, but has the drawback in that it relies on the repository owner providing such an interface. While most projects do have such a history viewer, especially smaller ones don't. What am I using a history for? It's mostly 2 things:
  1. understanding where code comes from. Sometimes there are little if's or assertions that are neither obvious nor documented but easily understandable when you see when and why they were committed (especially the commit message helps.)
  2. monitoring changes between two different versions of a file. This can either be between two timestamps ("X stopped working, who's fault is that?") or between two branches ("Why does it work so much nicer over there?")
  It would be nice if what ViewCVS can do were possible with an application on my local machine, ideally between different archives (my local arch branch of the main CVS repo or whatever).
• Don't invent your own $X when there's already a solution that's accepted out in the wild. This especially strikes me with arch's ChangeLog handling. I like the ChangeLogs in GNOME, don't screw me with those weird other ChangeLog method. It produces badly readable files. (see Rhythmbox's ChangeLog)
• It'd be nice if I could browse different projects without the need to download the repositories first. "Look at how KDE does it" is not that rare. Sure, I can use VIewCVS - if it's there.
• Another feature that I often miss is undo for every operation I do to my repository. Treat my repository like a Gimp image, spreadsheet or something, and have an undo button just like they do. So I can get my accidently deleted files back, undo the stuff I worked on for two days that doesn't work and whatnot. Shouldn't even be that hard to implement once inotify has hit the roads.