30 Sep 2003 mikehearn   » (Journeyer)

My last blog post pondered what was needed to share objects between "platforms" (ie .NET, C, Python, C++/std, C++/Qt and so on). This time, I'll be thinking out loud about potential systems that we could use in free software land. It'll be long, humour me. I promise I won't do it again ;)

I think there are basically 3 different types that stand a chance of being accepted. I'm going to exclude CORBA from this list on the grounds that it has already have a good attempt at the market, and failed to make a noticeable impact. So:

The first type is some derivative of COM

COM has been endlessly imitated throughout the years - because its core is so simple, it's easy to duplicate. For instance, Mozilla uses XPCOM, which is a simpler, slightly cleaner variant (but is designed for internal usage only rather than operating system wide usage). From the XPLC web site, I'd guess that's similar to COM also. The basic features of a COM style system are that you pass around arrays of function pointers (an interface), and each object must expose at least one standard interface, typically similar to IUnknown (XPCOM uses nsISupports).

I don't think a COM derivative is right for us. In order to be truly useful especially for performing language bindings many things must be built on top, like for instance a variants API. Because all this must fit in the basic framework of cascaded function tables, it's difficult to get a natural API. Microsoft themselves are leaving COM behind in favour of .NET - it would be foolish to head in a direction the dominant technology provider is leaving. COM systems often require free floating external magic, for instance a registry somewhere. Getting people to agree to depend on this would probably be difficult - the "what's in it for me?" factor gets in the way. While COM would work, it'd not the best we can do. There is also no widely deployed implementation available - having working code on peoples systems is a great boon. A COM style system does have the big advantage of being a known quantity, and at least to most politically acceptable if not technically so.

The second type of system is a common language runtime

.NET provides a CLR and CTS (common type system). As far as I know, .NET is the first attempt to produce a truly generic compiled ABI - traditionally they have always been designed for a particular language.

A quick review - .NET allows language interop not through making languages "skins" of C# as some have cynically suggested, but by providing a unified binary format and type system. Managed C++ extends C++ for instance, but it's still possible to compile the entirety of Microsoft Word with it. Some people seem to think that if a language has a form that can be compiled to MSIL it must be "crippled" - in fact this is no more the case than a language being crippled because at some point it must be translated into x86 opcodes.

The "crippled" accusations normally arise because people see that in order to share objects with other .NET languages they must conform to the common type system. This much is obvious - only C++ supports multiple inheritance so an API that makes use of this feature cannot be used from a language that does not support it. That doesn't mean you can't use multiple inheritance in .NET, it just means that objects you create using it will only be usable by other Managed C++ programs.

Clearly, in order to share code you must have a common ground, this was covered in my previous posts. The .NET CTS is an extremely rich common ground, and therefore .NET APIs can be very expressive. Because the .NET class hierarchy is available to all .NET langauges that understand the CTS, it's possible to write an API in say Python that returns an XML document usable from C#, as long as they use the common ground provided by that shared class library. You're free to write and use APIs that work in terms of PyXML of course, but don't expect to be able to export that to other languages. Sharing code between platforms is always going involve a set of compromises, but this one is unusually good.

The approach .NET takes to allowing code interop is technically elegant, and makes the most sense. Creating a rich shared ABI, object model, a strong introspection framework for interpreters and also providing a shared class library is probably the ideal way to enable code sharing at the technical level.

We now hit a problem

Creating all this infrastructure takes a huge amount of effort. It's taken Microsoft years to produce .NET - so far there is no equivalent being produced by the open source community. No, Parrot is not it - even assuming they overcame the technical problems with some see with their designs, it still only provides one part of the puzzle, namely a VM and new opcode/binary format. As far as I know, Parrot does not provide a shared object model, type system, and it most certainly does not provide a shared class library. Parrot is not specified anywhere, it's magic, - by contrast, the core of .NET is specified in ISO standards.

It could be done of course, by a team sufficiently dedicated. But, what you'd end up with would simply be something very much like .NET but incompatible - where is the logic in that? It makes more sense then to clone .NET exactly, and remain compatible with Windows - something we need anyway if we are to run many of the apps of the future. For this of course we have Mono, and a very important project it is even if you really don't like .NET at all. It's the Wine of the year 2010.

Mono, of course, has understandable political problems. Remaining compatible with .NET has a tangible cost, both technically and socially. It not only causes wierd hacks and warts like using the PE format for binaries instead of the more common on Linux (and somewhat superior, imho) ELF format, but socially it places control of the shared platform in the hands of a sworn enemy of not only Linux but the entire free software movement.

It's not surprising that people question the wisdom of writing free software for Linux on this framework. Don't get me wrong - patents do not enter into my argument here - rather, I'm more concerned about the cost of having Microsoft dictate the design of the framework and the statistical difficulty of "forking" it - we need Mono to be compatible with MS.NET for the apps, so we cannot simultaneously take it in another direction if we disagree with what Microsoft are doing.

Is there are 3rd alternative?

Of course there is. It's not perfect, far from it. It's not as wide-reaching or elegant as .NET, it's not as simple or well understood as COM. It is, like all attempts to bridge different platforms, a set of compromises.

It's GObject, or rather, a derivative of it. Most people here probably see GObject as a way of bringing object orientation to C, something already done by C++ and Objective-C years ago. In fact, one of the primary motivations for developing GObject was the need for a way to make language bindings very easy to do, and at this task it succeeds admirably. Often the parts of GObject that seem unnecessarily complex such as the dual phase destruction process are there to support language bindings - in this case, dual phase destruction allows garbage collectors to break up reference counting cycles before destroying/freeing the individual objects.

GObject is not especially fun to define objects in. It typically requires a lot of boiler plate code, a lot of tedious typing and so on. This is not a valid reason for avoiding GObject in our quest to share code. The need to write lots of code to define an object is only true in C, and by its nature, everything in C is tedious and involves a lot of typing. If you want ease of use, use Python, not C.

The reason GObject is valuable for sharing objects is that it provides a rich object model, type system and more importantly it exposes it all via a C API - accessing it from C++, Python, .NET and so on is easy.

Typically today, GObjects are written in C. There is no particular reason why that must be the case, in fact there is already a proof-of-concept program that uses .NET reflection to examine a .NET object hosted by Mono and spits out libmono->GObject bindings. You can then apply the PyGTK codegen code to bind it into Python, C++ and so on.

You can of course also go direct from .NET to Python, in at least two different ways. The first is to write a Python interpreter that spits out MSIL, and that is then JITed by the mono runtime. I think ActiveState were working on something like this. Unfortunately Python is largely magic remember - there are no guarantees that code runs perfectly unchanged when we do that. The other way is to do direct Mono<->Python bindings, ie that use the libmono/libpython C apis to directly map objects between them. This is possible because both platforms support good reflection/introspection capabilities.

What advantages does using C/GObject as our "hub" bring?

The first is that GObject is a very rich object model - almost as rich as the .NET one, I think. It supports things like signals, which typically have to be bolted onto a languages native object model (see libsigc++). It supports a variety of memory management models. It supports properties, static and virtual methods (kind of, see below), and it supports arbitrary string->value associations. You can do inheritance with it, in contrast to COM which does not really know about that.

The second is that the combination of GObject, GLib and C also provides a lot of the infrastructure needed to usefully share code in the form of a common type system, variants api, marshalling infrastructure, common data types and so on, which are frequently very minimal. There's no need to haggle over the definition of a list, because it's already been defined, and has only the bare essentials. The binding logic is free to layer on top any API it sees fit - translating a GList (which is little more than struct { void *prev, *data, *next }) into a QList which provides a much richer API is straightforward.

GObject has a few practical advantages. It already exists, which is a major plus, and better, is already deployed - every Linux (and FreeBSD and Windows) system that has GTK has GObject. It's well understood - GObject has been used to bind many types of objects into other languages, not just widgets or GNOME APIs - the GNU Triangulation Library, for instance. There is documentation, experience and tools available to make the job of producing the wrapper code easy.

The final advantage of GObject is that it doesn't depend on the user to set anything up. If GTK is installed, that's all you need. There's no registry, no runtime needed - as long as you can deal with shared objects, you're sorted. It can be used entirely internally, for instance you can write parts of a program in C and parts in Python.

GObject though falls short in a few places.

For instance, an object bound into Python from C will not respect virtual method overrides - GObject/GType knows nothing about virtual methods, and there is no set convention for them. A mix of styles is currently used, with no easy way to override methods as a result. I have some ideas for how that could be made better. The defs file format which expresses an API (similar to COM/CORBA IDL) is not specified anywhere, nor even really documented it seems. The tools to bind objects from C are plentiful but the tools to bind objects to C are virtually non-existant. Thread safety is not really handled, unlike in COM where thread safety is (rightly) specified as part of the objects interface (well actually its specified in the registry, not IDL) - though arguably this should not be a part of the object system at all.

Its last problem is that its name starts with a G. Yes, this is really sad, but it causes problems for some people. Hopefully the political problems (which really only seem to affect KDE developers) associated with "another desktops library" are more easily surmounted than the problems with .NET - at any rate, an extension of GObject to deal with the problems outlined above could be called, for instance, CoreObject: maybe that would help.

Latest blog entries     Older blog entries

New Advogato Features

New HTML Parser: The long-awaited libxml2 based HTML parser code is live. It needs further work but already handles most markup better than the original parser.

Keep up with the latest Advogato features by reading the Advogato status blog.

If you're a C programmer with some spare time, take a look at the mod_virgule project page and help us with one of the tasks on the ToDo list!