Older blog entries for company (starting at number 39)

Miguel, thanks for pimping Swfdec. But I think your conclusions are wrong. Flash is not even close to what I'd suggest as an option for "rich media".

For a start, the currently supported Video formats for Flash are Sorensen Squeeze and On2 VP6. As far as I know, both of those companies are actively fighting Free implementations of the codec. That makes Flash video as bad or worse than VC-1. You can play all 3 of them with FFmpeg btw.
Then there's the fact that the interaction between Flash and browsers is crap. You can execute Javascript from the Flash file and you can execute predefined Flash functions from Javascript, but there's no DOM or anything. But what about integrating Adblock, the "animate only once" option, popup blocking or all the other things you want integrated?
As for the things you mentioned: The Flash spec is semi-open, in the sense that you may not use it to write an interpreter for Flash. That's equivalent to closed for everything you care about. And the spec covers the file format only, not the video formats used or what script objects are supported.
And you can populate Flash files using remote data. You can even chose between XML and Flash. The XML needs to be interpreted, but I'm pretty sure there's convenience libraries for you that do that.

And it's not like the Free alternatives are far behind in the game. HTML5 has a video tag and SVG is far superior to anything Flash offers in drawing options. Especially because it's scriptable while Flash shapes are not.

There's 2 things missing:

• Simple animation support: Flash makes creating animations extremely easy. Doing the same thing in anything Free (HTML, SVG, even Gtk) requires being a good code monkey. The example above required NO coding.
• Authoring tools for rich web applications. Depending on who you ask, the best Free authoring tool is either vim or emacs. There's the Gimp and Inkscape of course, but neither does animations nor scripting, so it's pretty useless for doing rich web apps.

So I can tell you what you want. You want to create an awesome authoring tool that integrates HTML5 and SVG and make Webkit and Mozilla handle those. And after that I bet rich web apps are Free.

Talking about security is "funny". Like looking how people react to security issues or talking with other developers in #gnome-hackers, where I recently claimed that GNOME's security sucks and got quite a backlash. Since writing multimedia players like GStreamer or Swfdec made me think about security (I'm still far from expert unfortunately), I'd like to elaborate a bit on that.

Let me first mention one important thing: I don't just consider crashes to be stuff that shouldn't happen. Everything that disrupts the user's computing experience should not happen. Taking too many CPU ressources or too much memory for too long effects other applications negatively, so it should be avoided.

The biggest problem for me has always been the question of trust. When interpreting data I have to ask myself if I trust that data or not. If I have a number that says "allocate this much memory", I better trust it to report a correct size or I'll get an out-of-memory problem. A worse problem are pointers: read (or write to) the memory pointed to by this address. If you can't trust a pointer you'll likely get a segfault. So you better trust every pointer in your app. Note that this includes modifying pointers (like array indexing), which should only be done with trusted values. Stuff like that quickly gets my head spinning about which values are trusted and which aren't. Another example: Do we trust this string to be valid UTF-8?

A lot of data is read from files these days. Now, what files can application developers trust? Apparently we trust all the data a user doesn't have write access to, since I can make apps crash easily by removing Glade files, icons or linked-in libraries. But should applications trust stuff in the user's home directory?

Currently a lot of applications trust stuff in the user's directories unconditionally. Bash and X execute files in there for example. However epiphany allows you to save files from the web in there, too. What do you think clueless users would do when a web site tells them to download a file named .xinitrc? Probably nothing, since .xinitrc must be executable in order to get executed, but we're already very close to running "rm -rf ~" on login here. And of course we suddenly have a lot of untrusted files in the user's home directory. Nautilus for example knows this, since the thumbnailing process has brought down Nautilus quite often in the past. But I bet a lot of other applications don't think about that.

And then there's the fact that it's apparently dead easy to find security holes in Free Software while Windows products have been exploited by those issues in the past.
So I guess when the Linux desktop gets interesting for crackers, we'll be in for quite a ride. And by "we", I'm talking about every application that reads files.

So I've managed to write software that a lot of people are suddenly interested in. It's an interesting thing to say the least. However "becoming famous" was nothing like what I'd had imagined. Here's what happened after my initial blogpost:

1. The media picks it up.
   The media is interested in new stories. So they post them. However, they often prefer to give those stories their own spin. (The really weird stories have luckily already disappeared from Google, so no links to them.) None of those journalists contacted me about it, they just wrote something.
2. The people pick it up.
   Lots of people start talking about it. How do I know? I googled. Only 5 people talked to me about it. Mostly they just went into obscure forums or chats to discuss it and complain about missing features. Almost nobody of them realizes that it's a really good idea to come to me and ask questions. It's highly motivating if someone shows interest in my product. Even if it's just complaining that it doesn't build on his box. And if you ask, there's a high chance I get interested in solving your problem, especially if it's a pretty simple problem.
   (Side note: The Ubuntu forums are lucky to have such a high Google ranking, so I could at least find their questions.)
3. When there's a release, the distros pick it up.
   Distros are really fast at taking your source code and sticking it in the unstable branch of their package repositories. Lots of them, in particular the big ones, contact you to tell you about it and ask questions. The process of integrating new software into distros seems to work really well. Congrats from me to the distro people for that. Great work you're doing there. In particular Gentoo.
4. People start blaming
   They seem to read everything I have said in the past and spin it in their direction. I'm suddenly personally responsible for not working on Gnash and fragmenting the Free Flash movement, I sure as hell have to provide OpenBSD support or my software is crap, stuff like that. Again, noone comes and asks, that would be far too complicated. And probably wouldn't be compatible to their predetermined opinion.
5. Hackers don't show up
   I was expecting some interested people to show up and have a look at the code. In particular because the code is documented far better than all my previous code (including GStreamer ;)). I think there have been 2 people so far that looked at it. But that's about it. Probably everyone is busy doing their own project. Or people still think doing Flash is hard (hint: it's not).

But then, the few people that have shown up and talked to me have made a lot of things happen and I guess I'll cut a new release soon to make those people happy. In particular:

• People wanted an easy API to embed Flash files. So I created libswfdec-gtk. You should be able to play Youtube videos in your Gtk app with 20 lines of C.
• Or you could do it in 10 lines of Python with the new Pyswfdec Python bindings that Gian Mario Tagliaretti has been working on.
• Last but not least I've managed to use GStreamer instead of ffmpeg/mad for video and audio decoding. While that doesn't sound to exciting at first, it means you can build swfdec now without linking to patented multimedia formats and still be able to enjoy Youtube. I know at least Fedora was very interested in that.

No new work on Flash features so far, since I've been working on a branch redoing the ActionScript interpreter. Once that hits (that'll definitely be after next release), it'll make a huge number of Flash files supported. But that's still some work.

What I've been wondering and like to get some feedback from people active in this area is if it's a good idea to export the script interpreter. ActionScript is almost the same as Javascript (same syntax), so it offers a sandboxed execution environment. In this case completely GObject'ified. So you could use it if you wanted to script Flash files, or you could even add hooks to it in your own app and download interesting scripts from the web. Think Get Hot new Stuff with scripts. To me that's somewhere between total crack and really exciting.

It's done. Swfdec 0.4.3 is out, so for everyone that couldn't or didn't want to dig into the dirty world of unreleased code, grab the releases of Swfdec and Swfdec-Mozilla. It will probably take a while before this hits the big distributions, since there's legal issues with the video plugins and Debian and Ubuntu are still busy releasing.

For this release I've done all the easy tasks in Youtube playback, so you can play/pause, rewind and watch the slider on the seek bar. You still can't seek or change the volume, but that has to wait. But you can right-click, select Properties and save the video you're watching. So there, a feature that's not in the official player.

A big thanks to all the people who've encouraged me, filed bugs or submitted patches, wrote articles or blog posts about swfdec and otherwise kept me busy for the last week. It's been an interesting experience although I still think I suck at marketing and public relations. Somehow Christian does a much better job pimping my stuff.

The only thing I've figured out is that even though lots of people are interested in Flash in many different forms, there's a very small amount of people actually hacking on making Free Flash a reality. Even though the technologies used in Flash are what everyone wants. I wonder why that is?

So here's a small update to last week's Swfdec Youtube post. I've spent most of the time on the awesome IRC feedback that I got. Thanks everyone. Here's some answers to the most common questions I got:

Swfdec plays Youtube. Now what does that mean?
It means that when you grab the Swfdec library and the swfdec-mozilla package out of git and manage to install it correctly, you will be able to go to any Youtube video site and have it play back the videos in your browser. The buttons don't work yet and it certainly doesn't behave 100% like the Adobe plugin, but it certainly plays the videos.

OMG, swfdec doesn't work on 64bit machines!
OMG, it does! Since I posted the last entry, I've got lots of offers for remote access to 64bit machines and spent the last hours making sure it compiles and passes the testsuite. Of course I have no 64bit machine and until 3 hours ago didn't know most of the things I'd need to look for, so expect some bugs being left. Please test if you're on 64bit and if you find crashers, file bugs.

Where's the release?
I intend to do a release soon, but I'd like to have a working pause button and slider in Youtube. I'm hoping for a release at the end of this week. For now, use git.

How close is Swfdec to being a complete Adobe Flash replacement
That really depends on your definition of close. For the definition "implements all of Flash's features" it'll probably not hit 5%. For the definition of "plays all the Flash files on the Web" I think it's 80/20 right now. Swfdec plays 80% of the ads and 20% of the real content. And no, that wasn't on purpose. I've been following a simple rule for what to implement: I take an interesting flash file and make it work correctly. So if there's any Flash files that I might find interesting and that you want to have working in Swfdec, don't hesitate to contact me with URLs to those files. I'm always looking for cool Flash files.

What can be expected from Swfdec in the future?
I have no idea. I'm looking for a job at the moment and since that job will most likely not be me hacking on Swfdec fulltime, expect the Swfdec development to slow down in the future. However, it only took me half a year to get Youtube working and Youtube is already quite complicated. So in short: I have no idea.

YOOOUUUUUTUUUUUBE

GO Swfdec!

6 months of work, I'm now gonna go partying.
See you at Cebit.
PS: For the brave people: The code is in git.

Miguel links to a blog post by Anne Zelenka (read the comments!) about how open Flash is and how good an open Flash would be. I think one point they all don't see in their "an open spec would be enough" is the work necessary to produce an open spec.

Now let me get this straight. Adobe has a spec for the SWF format, and they have lots of documentation on Actionscript, so producing some spec would be easy. But the problem is that the existing specs only describe correct behavior, but not the more important part on how to treat errors. Consider an example where the spec might say something like "height: Integer - the height of the current movie". So what happens when your code does height = new Object()? It's written down nowhere. In the current closed world, the solution is easy: the accepted behavior is what the Adobe Flash player does. So if you want to write an open Flash Player like Swfdec, you don't need a spec, you need patience and lots of test cases. Because there'll surely be a Flash somewhere that does height = new Object () or height = height / 0 or height = "Hello World" or...

A good example of how hard it is to handle the unexpected right is Acid2 for CSS. And in the case of Acid2, there even exists a spec about how to handle all the errors. (I was going to link to something I read by I think Håkon Lie about why having a defined way of handling errors is important as opposed to just aborting, but I can't find it in Google.) And error handling mechanisms are an important part of an implementation. The Mozilla team needed 1.5 years to correct their error handling. So if you figure out something new and exciting about Flash, it can easily mean you have to redesign a large part of your player. So it's important to know beforehand and should be part of the spec.

And while we're talking about necessary rewrites: A part that no spec talks about is implementation complexity. If a function is O(1) in the official player while it is O(N) in yours, you have a problem when someone calls it excessively in a loop. And then there's probably code relying on timeouts, data input or the phase of the moon.

Another thing I've been wondering about lately is the complexity of implementing a standard. I have no clue how Flash relates to SVG in complexity, but SVG has an open spec and that one is 4 years old. Do we have any SVG compliant implementations by now? HTML has a free spec, too. It took the Mozilla team 6 years from open sourcing to a release for their browser. So if a complete Free Flash specification started to exist tomorrow, would it take 5 years to implement?

An open Flash spec would definitely make Flash inch closer to World Domination, but it'd still take a very long time to make it really Free. Open sourcing the player would probably make that happen way faster.

I did it

So I've just managed to finish my Christmas present only a week late, but anyway, after 3 months of not hacking on anything else, I've finally released Swfdec 0.4 and its companion, Swfdec Mozilla 0.4.

The good news: The mozilla plugin is now almost 100% compatible with South Park Studio. So you can do this. (Yes, that's an SVG taken from the Flash)
Unfortunately there's no UI for it yet. Implementing Flash is kind of a huge task afterall.

And finally, here's the disclaimer: This software is not rock-solid. So if the plugin kills your browser, don't blame me. File a bug or talk to the list instead.

Happy new year!

So, talking about the MS/Novell deal has been declared stupid by dobey even though according to other hackers 80 percent of hackers think that there is nothing wrong with it. jclinton has issued a statement condemning this action. Perhaps he said it best

"Since you guys are all buddy, buddy and everything, I'm sure it wouldn't be too hard for you to take a few minutes of his time to explain the spirit of Free Software to Steve."

Personally I think that if people weren't outraged about it, we'd have already lost to MS.

Update: Iain has said that if more people think like this he will do what he is told and stop blogging completely.

Flash

So about a month ago Federico blogged a Youtube link again. That made me wonder about Free Flash decoders and I had a look around. In the end I started hacking on David's Swfdec, since it has all the goodies: Cairo for beautiful Graphics, GStreamer for sound, David's excellent code (the SWF parsing code is awesome) and of course it was written in Gnome C.

Long story short: I haven't had this much fun in hacking for ages. I think I haven't done much else in the past month but hacked on Swfdec and watched lots of stupid Flash movies lots of times.

So what's the current state? It's still very far from finished since Flash is huge. It's a graphics framework (think GDK) with a scripting engine (think Javascript). But if you're adventurous, bored or both, feel free to check out the sources from git://people.freedesktop.org/~company/git/swfdec and play with them. They might even play back some weird flash file. For now I'll continue to make FLash files work one by one. Next stop: South Park.

PS: Before someone asks: Gnash is low quality, written in C++ with Boost, requires Copyright assignment, svn doesn't even compile and it does less than my current Swfdec, so I didn't hack on it.