11 May 2010 conrad   » (Journeyer)

Streaming Ogg Vorbis with sighttpd 1.1.0

I just released Sighttpd version 1.1.0, which includes support for streaming Ogg Vorbis from standard input. In an earlier post introducing a new HTTP streaming server (sighttpd 1.0.0), I described how sighttpd could be used to stream raw data, such as plain text:

$ while `true`; do date; sleep 1; done | sighttpd

and H.264 elementary video streams but not Ogg, because an Ogg stream needs to have setup headers prepended for each codec stream. "Instead, we would need to do something like Icecast: buffering these headers and serving them first to each client that connects before continuing with live Ogg pages".

So, that's exactly what version 1.1.0 introduces with a new <OggStdin> module. The sighttpd.conf setup is similar to the normal <Stdin> configuration:

Listen 3000

# Streaming Ogg Vorbis from stdin, using the special
# OggStdin module that caches Ogg Vorbis headers
<OggStdin>
        Path "/stream.ogg"
        Type "audio/ogg"
</OggStdin>

You can run this with a shell pipeline like:

$ arecord -c 2 -r 44100 -f S16_LE -t wav | oggenc -o - - | sighttpd -f examples/sighttpd-oggstdin.conf
And you can connect to it as an Ogg stream, eg:
$ ogg123 http://localhost:3000/stream.ogg

At the start of an Ogg Vorbis file or stream are three mandatory header packets:

  1. The Vorbis BOS (beginning of stream) header, which describes basic information like the number of channels and the samplerate of the audio.
  2. Metadata in VorbisComment format, which basically consists of text values like "ARTIST=Richard Feynman".
  3. The setup header, which includes "codec setup information as well as the complete VQ and Huffman codebooks needed for decode".

We can view the raw contents of these packets with oggz dump:

$ oggz dump Kobi-Birk_20011125.ogg |head -n 30
00:00:00.000: serialno 0639825516, granulepos 0, packetno 0 *** bos: 30 bytes
    0000: 0176 6f72 6269 7300 0000 0002 44ac 0000  .vorbis.....D...
    0010: 18fc ffff 00f4 0100 18fc ffff b801       ..............

00:00:00.000: serialno 0639825516, calc. gpos 0, packetno 1: 94 bytes
    0000: 0376 6f72 6269 7320 0000 0058 6970 686f  .vorbis ...Xipho
    0010: 7068 6f72 7573 206c 6962 566f 7262 6973  phorus libVorbis
    0020: 2049 2032 3030 3130 3831 3303 0000 000a   I 20010813.... 
    0030: 0000 0074 6974 6c65 0042 6972 6b0b 0000  ...title.Birk ..
    0040: 0061 7274 6973 7400 4b6f 6269 0d00 0000  .artist.Kobi ...
    0050: 6461 7465 0032 3030 3131 3132 3501       date.20011125.

00:00:00.000: serialno 0639825516, granulepos 0, packetno 2: 2.820 kB
    0000: 0576 6f72 6269 7325 4243 5601 0040 0000  .vorbis%BCV..@..
    0010: 8020 9a19 a7b1 945a 6bad 1d72 9a42 abb5  . .....Zk..r.B..
    0020: d65a 6bad 2594 5a5b adb5 d65a 6bad b5d6  .Zk.%.Z[...Zk...
    0030: 5a6b adb5 d65a 6b8d 81d0 9055 0000 1000  Zk...Zk....U....
    0040: 0021 0c55 0651 c99c d65a 6b44 1064 0649  .! U.Q...ZkD.d.I
    0050: e920 d65a 6be8 a0a5 105a 4cad d65a 6bad  . .Zk....ZL..Zk.
    0060: b5d6 5a6b adb5 d61a 6320 3464 1500 0004  ..Zk....c 4d....
    0070: 00c0 1863 8c31 0619 6410 5248 21a5 9452  ...c.1..d.RH!..R
    0080: 8c31 e618 74d2 5147 9d76 da71 6821 9594  .1..t.QG.v.qh!..
    0090: 5acc 2de7 9c73 ceb9 d61a 080d 5905 0024  Z.-..s..... Y..$
    00a0: 0000 a838 8664 5886 0584 86ac 0200 3200  ...8.dX.......2.
    00b0: 0004 1024 4353 34c7 d554 cf34 5d55 0542  ...$CS4..T.4]U.B
    00c0: 4356 0100 4000 0002 8000 0a18 4451 1445  CV..@..... .DQ.E
    00d0: 5114 4551 1445 d1f3 3ccf f33c cff3 3ccf  Q.EQ.E..

When a client connects to a stream somewhere in the middle of a song, these headers from the beginning are required in order to decode the audio data. sighttpd writes the pages containing the 3 header packets to a temporary file (created with mkstemp(3)). When a new client connects, the contents of that file are sent to it with sendfile(2) before jumping into the current contents of the stream.

I'm not trying to make a replacement for Icecast, but instead building a more general streaming server -- and of course I want it to have good Ogg support! So, please try it out, and leave some feedback in the comments or in email to me or ogg-dev :)

New Advogato Features

New HTML Parser: The long-awaited libxml2 based HTML parser code is live. It needs further work but already handles most markup better than the original parser.

Keep up with the latest Advogato features by reading the Advogato status blog.

If you're a C programmer with some spare time, take a look at the mod_virgule project page and help us with one of the tasks on the ToDo list!