Older blog entries for rbultje (starting at number 14)

Surround sound support in GStreamer:
Some genious mind provided me with a surround sound system for my laptop. I just got it working, and my housemates will hate me for quite some time now. For some unknown reason, I know have six speakers in my three-by-two meter room, and they make enough noise to keep the whole house with me. Lovely!

Interestingly, we also pretty much figured out how we want this to be done in GStreamer. My previous proposal was not ideal enough, so we adapted it slightly. It now is somewhat more extensible than the previous proposal.

Basically, for a GstCaps/GstStructure with X channels, the structure will have an extra property "channel_positions" (if absent on channels <= 2, it's simply mono or stereo), which is a GValueArray filled with an enum of type GstChannelPosition. This will proove to be a truely adventurous task, because we haven't used named enums before. Particularly, serialization/deserialization (going to enum from string and back, for things like GConf or gst-launch commandline and so on) will proove to be interesting. Secondly, we will need type-specific (extendible) fixation functions (to make sure that it fixates to something sane (which somewhat resembles "surround", rather than "all channels topleft".

The enum will be similar to Microsoft's enumeration of channel positions, as specified here.

Time to proove myself worthy to this genious mind! I'd better have some interesting 5:1 DVDs with me for the next few weeks! :).

The instructions are: Grab the nearest book, open it to page 23, find the 5th sentence, post the text of the sentence in your journal along with these instructions.

Due to there being 64 (4^3) possible codons, but only 20 different amino acids, the genetic code is degenerate: each amino acid is specified on average by about three different codons.

It's 10 PM, and I'm at the university. What a life...

So, GStreamer plays back .mp4a/.mp4v files now in CVS. Beat that, Fluendo gang! :).

And now that we've got that figured out, it's time to start working on multi-channel sound support. I recently wrapped dtsdec in GStreamer, and that makes up to a total of three codecs now (AC-3, DTS, MPEG-2/4 AAC) that support it. So far, we always downsampled to stereo or failed (in the case of faad; bad!).

So, what's needed:

• a GstCaps property for channel positions
• respective decoder plugins spitting these out
• alsasink checking for those
• audioconvert for channel conversions

So I'm basically doing the first and second now. We'll start going for a simple approach, similar to what Xine, liba52 and dtsdec do. They all use enumerations for different output modes (5_1, 4_0, stereo, mono). We will use a string that represents the number of channels in each part of your audio room (front, side, rear, lfe). The string basically consists of number-letter combinations, so 5:1 (3 front, 2 rear, LFE) sound will be "3f2rlfe" (f being front channels, s side channels, r rear channels and lfe indicates the subwoofer channel). The layout of samples in buffers is then interleaved, left-to-right, front-to-back, then lfe, so in 3f2rlfe, it would be left-front, center, right-front, left-rear, right-rear, lfe and then the next left-front sample. This isn't covering all corner cases of odd audio setups (there might be people with only three boxes on their left side), but will do for the general case.

The next step is to implement this in a52dec (AC-3 audio decoder plugin), dtsdec (DTS audio decoder plugin) and faad (MPEG-2/4 AAC audio decoder plugin), which is what I'm currently working on. Alsasink/audioconvert will be done at a later step (somewhere next week or so) and then all this should work.

So, anyone wanna buy me a 5:1 audio setup for my laptop for testing purposes? ;).

I'm - as of today - the new GNOME-media maintainer, together with thomasvs. I started my new task by sending an e-mail to gnome-multimedia@ to see what's up for GNOME 2.8. I personally would love to see Totem going into GNOME-2.8. We'll see... In the mean time, it'd rock for people to add bug reports for each of their files that fails to playback in Totem/GStreamer (see bug #138435) so that if we get this far, we'll support as many playback formats as humanly possible. :).

I'm also tinkering with the idea to apply for Cornell University grad school. I have a fair chance to get in, but I'll be gone from home, (old?) friends and family for five more years. Do I want that? Manhattan is still cool, though. Went out to see Slipknot, Fear Factory and Chimaira the other night. Nice stuff!

Miguel, you're not installing 160 MB GStreamer (source! That's only a few MB binary - nice trick ;) ) for playing MP3s. Or did you think Microsoft DirectShow only plays WMA audio? :).

GStreamer
How cool will 0.8.0 be! :). Honestly, it'll rock. I've been testing Totem over and over again for the past few weeks. I can playback a large amount of movies, pretty much anything on my harddisk. I *am* still having issues, but for all, the problem is known, so it should be fixable easily:

• One MPEG-4 video/audio Quicktime file doesn't decode correctly. That's because we don't initialize the extradata member in AVCodecContext using the 'esds' atom contents in the Quicktime file. Same for MPEG-4 AAC audio, but there, it's not required.
• The Quicktime demuxer currently has some issues with raw PCM or MACE audio. the problem lies in the demuxer, not a decoder or setup or anything. Will probably be a minor quick fix.
• Some MPEGs with highly unsynchronized elemtary streams will 'lock' our internal clocking (video far ahead of audio, but no fresh audio data). The correct fix is probably to make the amount of buffered data somewhat larger for output queue elements, say from 1 second to 5 seconds (those are limits).

Apart from that, I can playback most MPEG files, all other Quicktime files and with no single problem all AVI, ASF and MKV files. Please add bugzilla entries (GNOME bugzilla, product 'GStreamer') for every single "common media file" (so don't bother adding WMV9 movies ;) ) that doesn't playback using Totem-GStreamer using current CVS of both. Maybe I'll push some more and try to get GStreamer 0.8.1 out of the door for GNOME-2.6.0, with each of those fixed. That would rock! I didn't test Rhythmbox as extensively, but I know that I can playback MPEG-4 AAC, Ogg/Vorbis, MP3 with no problems. That's about 100% fo my collection. I don't use WMA (which should work), WAV (blah) or FLAC... I have some test files for each, though. I need more video test files.

GNOME
I'm - as of today - a happy member of the GNOME foundation! Yay! GNOME, luv ya!

U.S.A.
Did I mention how much I love the U.S.A., or more specifically Manhattan, so far? It's so clean, it's so controlled, it's so... Special! I simply love this place. Cornell University is a wonderful place to be at, too. Awesome!

ALSA support in GNOME Volume Control: yes, it's there! I finally pushed myself into installing the ultimate evil onto my computer ([*] yes, that's ALSA), and (helped by Jan Schmidt) added some related stuff to the ALSA plugin to get GNOME Volume Control to recognize it correctly. Now everyone can sleep peacefully again, gone's all regressions.

Did I mention that it looks horrible? See screenshot. Ohwell, we're in UI freeze, I'll rewrite the GNOME Volume Control in GNOME-2.8. :).

[*] ALSA's indeed the ultimate evil [**].
- it hides POSIX using something hideous called alsalib.
- it actually forces the use of this hideous thing by not documenting it's kernel API and putting kernel driver bug workarounds into alsalib.
- it does assert()s on errors (how about nicely reporting an error back to the application so it can notify the user? How about standard errno? See GNOME bugzilla #134007).
- It "transparently" does software conversion. This is cool, except that it shouldn't be the default. Unfortunately, the default device enables software conversion (default = plughw:0), thereby stimulating lousy application development.
- I feel like trolling.

[**] Next to terrorism, of course... See below.

--

Serious things: what can I say about Madrid? Nothing, I guess... Can I cry? No; this is beyond anything I can grasp... I'm currently in New York and visited Ground Zero last week. Why do people do this?

So, yesterday I finished autotoolification of ffmpeg. I sent Dave Schleef a patch (so he could do some assmbler magic to ffmpeg), and I've already integrated it (well, "sort of"...) in our new gst-ffmpeg module and am now watching Quicktime movies with my GStreamer installation.

So far for the good news. Bad news is that I didn't commit it yet.

I've been thinking about reviving our ffmpeg demux wrapper, which could be used as a fallback if we don't have one ourselves. That'd give us instant demuxing support for +/- 10 new exotic movie formats... Anyway, long-term...

Other stuff
I'm off for Evanescence (playing tonight at Roseland, NYC) in a few minutes, and am currently finishing up details here at the lab...

Short "hi" from Manhattan, New York City! I arrived here Sunday with a broken laptop, no internet or email access and a taxi driver that didn't speak english (oh my, the odds). I've been to Cornell university for a few days now, I'm getting accustomed to the city and generally, I think this city is excellent.

I hope to get a new laptop screen next week, have internet by then too, and get back to GStreamer development! For now, I'm off to the parties today and tomorrow.

Today is my last day at work. I'll have the next week off and after that, I'm hoping to live in Manhattan, New York City, to finish my studies (biomedical sciences) at the Weill Medical College (Cornell University) there. I can't wait to go there. But I know I'll leave a lot behind me for the next six months. I'm not expecting to have much time left for GNOME or GStreamer for the next six months... And I'll definately miss my friends here. :-(. We watched the movie "Discovery of Heaven" together yesterday, sort of a "last goodbye" for me. I'm expecting to see them once or twice next week, but I'm already starting to get that feeling that I miss them, and I'm not even gone yet. Still makes you drop a tear...

On the bright side of things, I can say that during the last month (when I worked on GStreamer for work), a lot of good stuff happened. I mostly rewrote and finished GStreamer's Plugin Writer's Guide (which had been unfinished for almost 2 years now). I've done quite a lot of work in basic video encoding, recording and parsing (mostly DivX and MPEG) and I just had a lot of fun. GNOME media seems to work fine too, finally, so things are good. I'll continue working for the company while in New York, but only a few hours per week.