20 Nov 2003 mikehearn   » (Journeyer)

Ahhhhh, I came to Advogato because I wanted to be distracted from work, and here I am again, trying to avoid writing this History essay. I love you Interweb!

I started hacking on something for GStreamer last night. I hope it won't take too long, I get jumpy if I'm away from autopackage for a while, and I keep feeling I should spend all my spare time on this contract work for Wine. Nonetheless, it's kind of important that GStreamer is more robust - the fact that it works at the moment is mostly a matter of blind luck (on single processor machines).

Luckily dolphy wants to build a high end streaming server using it, which means fixing SMP breakage, which means making GStreamer finally use threads sanely.

I'm not working for dolphy unfortunately, but as a part of that and my recurring interest in low level kung-fu I am writing an inter-thread marshaller, or to be more accurate as no actual marshalling is done, a thread thunker.

I fiddled with using inline assembly, some interesting hacks with the stack, gcc extensions and so on for a bit until I resigned myself to the inevitable - code generation is the easiest way to get portable and easy to understand code. The way it basically works is that you place a macro call (?) as the first line of your function (so you need a C99 compiler), for instance:

void some_object_method(SomeObject *self, int param) {
THREAD_THUNK(some_object_method, self, param);
....
}

"self" is a GObject (or really any structure) which tells you which GThread owns the current object. If that thread != the current thread, the call is redirected to a stub which places all the parameters into a structure which is then pushed onto an asynchronous queue. We then block on a condition until the owning thread dispatches the function call via a stub which unpacks the structure. The proxy/stub code should hopefully be transparent. So far, so simple.

Issues I don't currently deal with - the code generator is non-existant, I'll prolly try hacking that tonight. If you call a thunked function and that function calls another, the second function will be executed in the owning thread of the first which might not be what you expect. Signals have the same problem, they aren't marshalled into other threads, so you can get callbacks on a different thread to the one you expect.

Typically that's not a problem but code which uses TLS slots (GPrivate in glib), for instance OpenGL, can break if that happens.

You can also get deadlock as there's no re-entrancy yet. If you have two objects owned by two threads (A and B), A can call into B which can then call back into A but function dispatch is not being done, so the threads freeze. Probably the simplest solution to this is to have the stub dispatch incoming thunked calls while waiting on the result of the previous one. I say "simple" but of course this can get hairy really quickly - we don't want to do what DCOM/Bonobo do and re-enter the main loop, or perhaps even dispatch any non-related incoming method call while blocking inside another one. The principle of least surprise tells me I should somehow restrict the scope of thunk re-entrancy, but I need to think it through more clearly.

Fun stuff, anyway :)

Latest blog entries     Older blog entries

New Advogato Features

New HTML Parser: The long-awaited libxml2 based HTML parser code is live. It needs further work but already handles most markup better than the original parser.

Keep up with the latest Advogato features by reading the Advogato status blog.

If you're a C programmer with some spare time, take a look at the mod_virgule project page and help us with one of the tasks on the ToDo list!