Older blog entries for company (starting at number 18)

Byzanz

It took me a while, but I'm now able to do useful things with it. Or said differently: Byzanz 0.0.1 is ready for everyone brave enough to play with it.

What is Byzanz?

Byzanz is a desktop recorder. Just like Istanbul. But it doesn't record to Ogg Theora, but to GIF.

Screenshots?

Here's one you can make with it:

The idea for that demo was taken from Richard's blog. He said he needed quite a while in Gimp for his pictures. I needed around 30 seconds. That link goes to a 1.6MB recording of the session I used to record the above demo.

Why GIF?

GIF is the best format I found for recording desktop sessions. The criteria I wanted were these:

• You see use it in any browser. Preferrably without plugins.
• It records a typical interesting desktop losslessly or at least in an acceptable quality and in an ok filesize.
• It is free.
• Recording works fast enough.

What's a "typical interesting desktop?"

• Short tech demos like the one from Richard or similar.
• Bigger tutorial style demos that are currently typically done in Shockwave/Flash. And I can't see those.
• "Do this to reproduce the bug"
• And my original reason was: I wanted to demo my ideas for animated themes like this one:
  

Why the name?

I couldn't come up with a better one and thought it's a clever pun on someone else and his application. ;)
And it's a name that's not yet taken. At least apt-file search byzanz doesn't return anything.

Update: There's been quite a few people testing it (cool!) and writing me mails about one thing I forgot to mention: Byzanz currently requires a running Composite manager on your X server to work. If you think you're brave enough to try this, see here for a good tutorial on how to make it work. Otherwise wait for me to fix it in a future release.

Havoc has a big rant about why the iPod rules and all the other players don't. This rant is probably correct for the US, but it fails for Europe. Because all his arguments are based on the fact that the iPod is the dominant music player. And in Europe, it isn't. When I'm having my daily commute in the subway, you can see some people with iPods, but most of the people use so called mp3 sticks. And of course I have reasons for why I believe the iPod fails in Europe:

• Because these sticks are dirt cheap (you can get them for less than $50 and even when they were you they cost less than $100 pretty fast). They are in fact so cheap, it's a convenience product. The iPod is expesive and definitely not convenience.
• Music players aren't cool (at least not with people I talk to). Mobile phones are. And conceerning the youth ringtones are definitely cooler than a whole song. Ican only imagine what an Apple mobile phone would result in...
• Apple doesn't really live up to the "all your music" claim. People here use Windows and Winamp or Windows Media Player. So using the iPod would either mean switching your music playback program or copying your songs manually in Windows Explorer. All your music?
• I don't know a single person that had even considered buying anything at iTMS (or at any of the other music shops that do far more advertising around here, like Musicload). People I know trade CDs and burn them and that doesn't work with iTMS purchased songs.

So I guess what it all comes down to is that I think that the iPod isn't doing so well because it's so well designed, but because Apple managed to make it synonym with "mp3 player" and especially "cool mp3 player". But it only managed that in the US.

When NASA was trying to go to the moon, there was a great deal of enthusiasm: it was a goal everyone was anxious to achieve. They didn't know if theys could do it, but they were all working together.

I have this idea because I worked at Los Alamos, and I experienced the tension and the pressure of everybody working together to make the atomic bomb. When somebody's having a problem - say, with the detonator - everybody knows that it's a big problem, they're thinking of ways to beat it, they're making suggestions, and when they hear about the solution they're excited, because that means their work is now useful: if the detonator didn't work, the bomb wouldn't work.

I figured the same thing had gone on at NASA in the early days: if the space suit didn't work, they couldn't go to the moon. So everybody's interested in everybody else's problems.

But then, when the moon project was over, NASA had all these people together: there's a big organization in Houston and a big organization in Huntsville, not to mention at Kennedy, in Florida. You don't want to fire people and send them out in the street when you're done with a big project, so the problem is, what to do?

Richard P. Feynman, What do you care what other people think?, p.214f., WW Norton & Company 1988, ISBN 0-393-02659-0

s/NASA/Gnome/
s/moon project/Gnome 2.0/
s/Houston/Utah/
...

I'll leave it as an excercise to the reader to figure out if NASA has gone uphill or downhill since reaching the moon.
The whole book this quote is from was an interesting read for me, especially with the parrallels between how NASA and Gnome evolve. And this quote sums it up best.

Says Rodrigo: These notifications expire a few seconds after they are shown, so they should not disturb the user at all.

The first thing that came to mind when I read this was that this line is the exact justification for popup advertisements in browsers. Some more advanced popups using DIV even expire themselves, so they meet pretty much the exact same criteria as notification popus.

I'm still surprised why so many people, especially those which do a lot of usability work in Gnome think it's a good idea to use popups. Popups do two things that have been identified as very bad in a lot of other contexts:

1. They occupy screen estate not assigned to them. To keep his screenshots as an example: The bottom right is where a lot of status bar icons in applications are.
2. They are animated. They're not containing animations, but the popping up and fading out of the notification is an animation. Unexpected animations cause interrupts in workflow. OTOH, Clippy is really cute, and you can switch him to look like Einstein.
3. Another thing not related to popups but related to libnotify is discoverability. If I'm away from my computer for 10 seconds I might have missed a notification.

So the question is how notifications should be handled? Well, there's different kinds of notifications:

1. Notifications the user wants to be interrupted for. Such notifications would be allowed and even should try to interrupt the user. Examples here are low battery on laptops, full hard drive, but also me waiting for a download to finish or a friend to come online and just browsing the web until that happens.
2. Notifications the user wants to look at later. This is mostly stuff where the user decided that things he's interested in haven't happened yet and then diverts his attention elsewhere, but will come back later to check. Examples here are downloading a movie someone linked to, checking email or friends going on/offline in IM.
3. Notifications that are uninteresting and the user will probably never want to look at. Examples here are all the join/part messages on IRC or IM, the temperature outside changing or the clock ticking.

I think these 3 categories pretty much sum up the different uses for notifications. At least I've not found any important steps missing. But feel free to enlighten me. An important thing to notice is that for different users different messages are important.

I think I'll now just sum up some ways that I have found for notification handling and how I feel about them.

• Windows XP pops up bubbles all over the place telling you things like "Where are my icons gone?" or "Where are my programs?", especially after installation to get people aware of the new features compared to older Windows versions. After I have installed XP for the fifth time, this is just annoying.
• Firefox (at least on Windows) pops up a little box in the top right to inform that all downloads are complete. Apart from the fact that it doesn't work correctly with fullscreen applications (read: games), it mostly fails to notify me of what I'm interested in: Most of the time it's the first download to complete, not the last. And there's the issue of discoverability. If I've been looking away for a short while, the box has already disappeared, so I might not notice the download having finished.
• All browsers (and also IRC and chat programs) use Icons or colors in the tabs to tell you when a tab has finished loading. This is a really nice notification, because it uses a space pre-assigned for this task and is easily discoverable. (Imagine libnotify would popup a little box for every page that finished loading... ;))
• Applications on Windows are allowed to flash their taskbar entry to notify. This is a nice feature, because it only occupies space assigned to them. Slightly bad things about this from my point of view is that it's constantly blinking, which tends to annoy me so I click the app to turn it off - I'd prefer if it'd just change the color of the taskbar and not blink and there's sometimes and issue with discoverability if I'm using an auto-hiding panel.
• Amarok uses an OSD for song change notification. While a funny little gimmick, it really gets annoying when something pops up on your screen every 3 minutes.
• Lots of applications have voice messages to signal notifications. Those are quite nice, unless I'm listening to something important (like when coding audio applications), don't want to be interrupted (like when coding), when I'm not at my desk (discoverability again) or when they happen far to often (Teamspeak's "player joined" message really is that annoying). In the end they're a great addition, but not a good mechanism in itself.
• Warcraft 3, Unreal Tournament and I suppose lots of other games too print notification texts pretty much centered and in a big font onto the screen. This works very well even though it's essentially a popup. The reasons for this I think are: 1) The message doesn't really occupy a lot of screen since the background is transparent and I only see the text message (slowly fading out), so I can easily see what's happening behind it. 2) The message is important, so I want to read it anyway. 3) There's no discoverability problem, because while playing I'm not away. 4) I don't perceive it as an animation while it's fading out, so I'll look at it when it pops up and can ignore it after that.
• Warcraft 3 offers a machanism to jump to the current notification. In Warcraft 3 this is a huge feature since it allows centering the view at where the action's at. Especially since the key is unused otherwise. The key is space btw.

So, summing it up, I think I like notifications that don't occupy screen estate they weren't supposed to use, don't divert my attention when they're not important to me and are easily discoverable even when I've not been there for a while. I have no clue how to ideally do that, but I know that the applet popups from Rodrigo fail on all 3 of these points.

The fun debate

I think the real reasons why it's not fun to hack on Gnome have nothing to do with the usefulness of the API, libs, platform or whatever. After all, Linus sounds like it was fun to write a kernel from scratch. There's so much interesting stuff to hack on (at least for me), there's no technical reason why Gnome hacking shouldn't be fun for at least the next ten years. No, I guess the real reason - and this has been touched by Seth , Martin and James without really noticing it: It's the community. It's how you feel when you communicate with the people the make up the project you hack on. And let me tell you one thing: The Gnome and GStreamer communities were great fun to be in, but they're less and less so.

I always compare the Gnome community some years ago to a college community, while the current community to me looks more like a corporate community. The names are chosen to show what the primary values for this communities are. You'll probably come up with better definitions than I can, so I'll just give some examples:

• Some time ago there wasSullivan. This was the last such big (typical college level) joke I'm aware of in Gnome. The file in that link was last modified one and a half years ago.
• I'm sure D-Bus is a cool technology, it's just that noone seems to care about it. At least there's noone on IRC that I could ask when I have a question. Yes, there's a mailing list, but mailing lists are not a good idea when you just have a small question. They are very bad when you have followup questions and they outright suck if you want to have a discussion.
• In fact IRC vs mailing lists is a good example of college vs community culture. IRC is a fast, informal and lossy (in the sense that you sometimes have to ask stuff twice) way to communicate with each other. Sounds a lot like talking at parties. Mailing lists instead are a slower, formal, archived and regulated way of communicating. Sounds a lot like filing taxes to me. Both are definitely necessary, but you decide what's more fun.
• A lot of discussion about directions, design and whatnot happens out of reach of the broader community. I'm currently witnessing this in the GStreamer project, where the next version is practically designed behind "closed doors". These closed doors are more often than not the office space of a company (in the GStreamer case it's really nice offices btw). But when half of the community is part of a company and somehow locks me out, there's not a lot of fun in it.
• If the most important thing about a patch is to be sure to not break ABI compatibility, it's certainly less fun than when the most important thing about a patch is what cool new features it adds. (I am fully aware that compatibility is important and I don't think that should change, but it definitely is less fun).
• The conference that represets the community costs money. Nuff said.
• The treatment of newcomers is interesting, too. College communites tend to welcome people with open arms. On IRC they are just integrated into the running discussion. Corporate communities tend to not have IRC channels and answer with one-liners on mailing lists. (See the D-Bus part above).
• Fun communities have fans.

When I first joined Gnome, it was fun and I thought "Hey, I'd like to do that for a living." These days with people even quitting their jobs working on Gnome, I'm not so sure.

What I can do and you can't

I put this in /usr/lib/gstreamer-0.8/python, run gst-register and have a new element available that does what it says.

import gst
import gobject

 class ChannelSwap (gst.Element):
  __gsttemplates__ = (
      gst.PadTemplate ("src", gst.PAD_SRC, gst.PAD_ALWAYS,
        gst.Caps ("audio/x-raw-int, width=16, depth=16, channels=2")),
      gst.PadTemplate ("sink", gst.PAD_SINK, gst.PAD_ALWAYS,
        gst.Caps ("audio/x-raw-int, width=16, depth=16, channels=2"))
  )
  __gstdetails__ = ("Channel swapper", "Audio/Filter", "swap left and right channel",
      "Benjamin Otte <otte@gnome.org>")

   def __init__ (self):
    self.__gobject_init__()
    self.srcpad = gst.Pad (self.__gsttemplates__[0])
    self.srcpad.set_link_function (self.link)
    self.srcpad.set_getcaps_function (self.getcaps)
    self.add_pad (self.srcpad)
    self.sinkpad = gst.Pad (self.__gsttemplates__[1])
    self.sinkpad.set_chain_function (self.chain)
    self.sinkpad.set_link_function (self.link)
    self.sinkpad.set_getcaps_function (self.getcaps)
    self.add_pad (self.sinkpad)

   def otherpad (self, pad):
    if self.sinkpad == pad:
      return self.srcpad
    else:
      return self.sinkpad

   def link (self, pad, caps):
    return self.otherpad(pad).try_set_caps (caps)

   def getcaps (self, pad):
    return self.otherpad(pad).get_allowed_caps ()

   def chain (self, pad, data):
    data = data.copy_on_write ()
    for i in range (0, len (data), 4):
      tmp = data[i:i+2]
      data[i:i+2] = data[i+2:i+4]
      data[i+2:i+4] = tmp
    self.srcpad.push (data)

 gobject.type_register (ChannelSwap)
gst.element_register (ChannelSwap, "channelswap")

And no, it's not performant. Now to write the mail with the (quite big) patch to the list, so you can do that, too. You can thank jdahlin for making me start hacking on Python btw.

Wanna puzzle?

Since rbultje promised me a coke, I present this to him and the rest of the world. (Screenshot! Go click the link!)
I'm convinced now that puzzling moving images is far harder than the standard image. If you like to play, the code is in gst-plugins CVS. If you also got a video player that forwards key presses (which buggy totem doesn't do here), you can just change your videosink to "puzzle ! xvimagesink" and start puzzling. If you're too puzzled, hitting space restores the normal picture. :)

If you're really hardcore, you watch a movie like The Matrix with lots of cuts and set your videosink to "puzzle rows=30 columns=40 ! xvimagesink" to get a puzzle with 1200 pieces. Maybe you'll even have solved it before the movie ends ;)

(updated, I was kind of in a hurry when I started writing it. And yes, I missed my bus.)

Since Colin and Havoc are into version control systems, I'll offer my thoughts, too. This is mainly what I realized when doing some of my code in arch (using tla) and some of it in CVS. It's also mostly evolutionary and doesn't try to go as far as Havoc's ideas.

• Please honour the structure of my project and allow me to use it. In CVS I can commit, diff, update and whatnot from any directory, with tla I need to use the top source directory.
• Don't clutter my working directory or invent other requirements. It is my working directory and I intend to work with it. This includes searching through it (tla puts lots of changesets or whatever into the .{arch} dir and these end up being duplicates in my searches). It includes moving the whole working directory elsewhere and continuing to work with it (tla caches inodes, so all hell breaks lose when I copy it to my laptop and try to do stuff from there). And it includes moving around files to better organize my source tree (not possible in CVS unless you move the files on the server).
• Allow changing all user-readable identifiers easily. I named one of my first repos for testing arch "company@localhost". When I later wanted to publish it, I couldn't just rename it to something sensible like "otte@gnome.org". Now everyone can happily use "company@localhost" for a while.
• Give me an easy way to work with a file's history. I'm currently doing this via ViewCVS mostly, but has the drawback in that it relies on the repository owner providing such an interface. While most projects do have such a history viewer, especially smaller ones don't. What am I using a history for? It's mostly 2 things:
  1. understanding where code comes from. Sometimes there are little if's or assertions that are neither obvious nor documented but easily understandable when you see when and why they were committed (especially the commit message helps.)
  2. monitoring changes between two different versions of a file. This can either be between two timestamps ("X stopped working, who's fault is that?") or between two branches ("Why does it work so much nicer over there?")
  It would be nice if what ViewCVS can do were possible with an application on my local machine, ideally between different archives (my local arch branch of the main CVS repo or whatever).
• Don't invent your own $X when there's already a solution that's accepted out in the wild. This especially strikes me with arch's ChangeLog handling. I like the ChangeLogs in GNOME, don't screw me with those weird other ChangeLog method. It produces badly readable files. (see Rhythmbox's ChangeLog)
• It'd be nice if I could browse different projects without the need to download the repositories first. "Look at how KDE does it" is not that rare. Sure, I can use VIewCVS - if it's there.
• Another feature that I often miss is undo for every operation I do to my repository. Treat my repository like a Gimp image, spreadsheet or something, and have an undo button just like they do. So I can get my accidently deleted files back, undo the stuff I worked on for two days that doesn't work and whatnot. Shouldn't even be that hard to implement once inotify has hit the roads.

threading

Everybody that has ever tried to use GStreamer in a multithreaded way knows that it is horribly broken. If you have a pipeline running in a thread and want to unref that pipeline in an EOS callback when it's done, you can't. You need to put an idle handler into your application's main loop and unref the thread from there. Ask the Rb and Totem guys about how intuitive that is. So what are the requirements for a threaded media playing framework? I've come up with these (in order): - The obvious part: Doesn't crash, no unguarded variables, ... - It doesn't deadlock unless the app does something wrong and that something is well documented - or better: obvious. - The pipeline has a minimal overhead when running in non-threaded mode. - Every element in GStreamer runs all of its code in the same thread. Since an element is in essence a vtable that's filled by the plugin, this means that all of those vtable entries are called by the same thread. (This is done for mainly 2 reasons: It avoids the need to lock inside elements, which eases plugin development and some stuff, like OpenGL, expect to run in the same thread all the time.)

Now here are things people want to do: - as mentioned above: unref a thread when it hits EOS - be able to change the running pipeline in a different thread (think gst-editor here) - be able to change properties of elements in different threads (think volume in a media player) - be able to listen to signals from a running thread (think error signal or volume-changed in a media player)

marshalling

After discussing this quite a bit the last days (thanks Mike for the input), I came up with the solution of marshalling calls to an element into the thread the element belongs to and dispatch them there. Block the calling thread until the call is dispatched and then continue. Of course this doesn't work for a simple example that even exists in current code:

• user changes volume
• GUI thread wants to set the "volume" property on the volume element in the gst thread. This is marhsalled into the gst thread, the GUI thread blocks.
• The gst thread changes the volume. This change triggers the notify::volume signal. The GUI thread has connected a signal handler for that to change the volume slider position. So the call is marshalled into the GUI thread. The gst thread blocks until done.
• Noticed something? Right, at this point both threads are blocked waiting for the other to be done.

I quickly came up with the idea to not just block a thread when we wait for a dispatch, but to allow dispatching of other threads' marshalling requests while blocking. This nicely works in my example code, but unfortunately, it has an important issue.

reentrancy

What's this and why is it a problem? This is easily answered by Google. Unfortunately reentrant marshalling would make the following GStreamer pseudocode wrong in the general case:

if (gst_pad_link (pad1, pad2))
  g_assert (gst_pads_are_linked (pad1, pad2);

Since gst_pad_link is reentrant, the pads could have been unlinked by other code that happened to be dispatched before returning. I'm currently investigating if this is a real issue, or if there are constraints that make it impossible for such things to happen. Another thing I'm wondering is how reentrant our code already is (g_signal_emit requires reentrancy for example - or do you know what happens inside signal handlers?) The last thing I'm wondering is how much of a burden this is for GStreamer's target audience: application writers and plugin coders. Since GStreamer itself doesn't do much unless you ask it to, most reentrancy issues should be caused by application code. Plugins only handle their own stuff and should never call out anyway. Or am I missing something?

I'm really not sure on the big picture here, since it's a very hard problem where I don't have much experience. So if you happen to have run into problems with this that I'm missing, feel free to mail me. And if you're one of the bonobo/Orbit guys that knows URLs to the reentrancy issues those projects had, so I can read up on it, get in touch with me.

life

I've quit my student part-time job. Writing Office macros annoyed me so much I didn't want to anymore. And since I still can't convince louie to make Novell hire German students working on GStreamer, I'll soon go workless. Sucks. But not as much as writing Office macros.

automation

I've just read Glynn's post about an article open source issues. While the article's main points is nicely refuted by Glynn, one line got me thinking: "If you can't install it, you don't deserve to use it".

My point: Free software tools are incredibly bad at automatically doing the right thing - or at least don't aid. GStreamer's support for "just working" is a mess - everything requires manual intervention when in fact the GStreamer core should just do the right thing. Examples are autoplugging (There's a "let's just hope nothing breaks" mentality), plugin registration ("it works most of the time and if not, just delete all your config files and try again") or selection of audio and video outputs ("Just try another possibility in gstreamer-properties"). Nautilus still doesn't present me with "network places", my Linux kernel requires manual insertion of the right modules (or alternatively, distro-specific discovery tools) instead of a tool that jsut does the right thing. Even my Debian packages have let me down recently, when I tried to install icecast2 only to figure out that activating it requires editing a config file which it tells you in a one-liner on bootup. Next time, pop up a dialog during install, please? And to finish it all off, gtk-doc is one of the worst tools in the just works department.

The reasons I have identified why this doesn't just work so far are these:

• We (the developers) all know how to get stuff done without automation. So why should we invest time in automation, when we can do without? I believe the fact that most people that are working on automation are company funded might be a good prove for this reason. Also there's a lot less duplication going on everywhere than in product automation. There's less mp3 players in Debian than distros available, which is another proof that automation (in this case installation) is tagged on and not designed into it.
• Automation is hard. It does not only require code that does it, but it requires knowledge of the full system you are working with and adding enough introspection capabilities to the system, so the code can work from within. And especially providing introspection capabilities is an often neglected part, if only because it's boring or people don't know how to do it.

Gstreamer autoplug

Of course there's also a reason why I started to think about such things and it's the new autoplugger I'm hacking on, that's finally supposed to do everything our old autoplugger missed. While I had to write a lot of ugly code to get my automation working, all the features have been implemented and I'm using it on my local Rhythmbox for playback without problems and without all the old workarounds. I think I'll do a first release soon to get more people interested in it, when I have the metadata importing working without the one hack it uses, too. Especially because that will add an important feature: It will automatically add all sound files GStreamer knows about and not just those that are "enabled". I'm already afraid of the day when I'll try to import / just to see if it works good enough. I hope to get this stable enough (including the video part) so we can retire the old autoplugger for the next major Rhythmbox and Totem releases and Gnome 2.8. Hopefully that'll be the first release where Rhythmbox doesn't need to work around any GStreamer issue anymore and just can play any file that has audio in it.