Older blog entries for lupus (starting at number 13)

The Nemerle guys point out that exception handling is not quite fast as they'd like in Mono and the other CLR implementations and assume it's a design issue of .Net.
I can't say about other implementations, but as far as Mono is concerned, we were busy writing a couple million other lines of code and exception handling speed is of little importance when compared to the amount of work we did and the amount of work we have ahead of us. That said, making some particular exception handling pattern work faster is easy and in fact I just committed a change that makes the posted test case 4 times faster. The trick is to either throw objects that don't derive from System.Exception or to throw already thrown exception objects: in both cases we don't need to get the stack trace info, which is what currently slows down exception handling.
Of course we'll speedup the code when we'll have more time to dedicate to fringe performance areas (the changes are not a 1-liner as the change just committed:-).
The code could be optimized more, for another 2x speedup, if someone is willing to dedicate the time.

Anyway, this issue reminded me of a similar issue that IronPython exposed: when Jim gave his talk at OSCon, there was an embarassing case of exception handling in the pie-thon benchmark where Mono resulted 100 times slower than CPython. A few days after we knew about the issue, I committed some changes to reduce the overhead to more sane levels (we are now about 8 times slower in that particular sub-benchmark and again most of the time is spent gathering stack trace info).

It's interesting to note how the Mono performance improved for the pie-thon bench since IronPython was released. In his slides he mentions Mono 1.0 was 23 times slower than CPython (2 times slower if the exception-handling bench is not considered). A few days later I posted at http://lambda-the-ultimate.org/ the numbers using mono from cvs (what was to become 1.1.x) and there were already some improvements, with mono 1.8 slower than CPython (without the dreaded b5 test).
Today I rerun the benchmark and the numbers have improved a lot: Mono is now 2 times slower than CPython including the heavy exception handling test. The nice related result is that mono is 10% faster than CPython if b5 is excluded. This amounts to a general 2x speedup in a little over 6 months.

This gets us back to the discussion about the design of .net/mono and exception handling speed. There is nothing in the general design that would make exception handling particularly slow, but of course the virtual machine is not built around the need to make exception handling fast, and for good reasons. As Jim quotes in his slides from the Modula-3 manual: ``Implementations should speed up normal outcomes at the expense of exceptions ... Expending a thousand instructions per exception raised to save one instruction per procedure call would be reasonable.'' One of the reasons IronPython could implement the Python semantics with good performance on Mono is because Mono and the .Net design allow for very fast method calls. Contrast this with, for example, the design of Parrot, where method calls and returns are built around the concept of continuations. This makes exception handling in parrot probably almost as fast as method calls, but normal calls are also made one or two orders of magnitude slower than in Mono.
We'll see more speedups in Mono's exception handling, but people should consider that exception handling will always be about two orders of magnitude slower than a simple method call (well, except for some special cases that we plan to optimize, where they'll be comparable:-).

Mono PPC JIT hacking
Yesterday was a sort of turning point: the mono JIT for PowerPC is ready for wider testing on both Linux and MacOSX. While there are still at least a few bugs, people are unlikely to run into them (but, of course, I expect the wider testing bit to turn on more obscure bug reports).
We can also run quite a few Gtk# programs (well, GtkDemo, the other samples and the monodoc browser: I'm sure some brave people will build and try MonoDevelop and report results). Especially important was testing the Fifteen Gnome canvas game, which I did throughly (I just needed to remember to run ssh -C, since I tried it from Miguel's G5 on the other side of the Atlantic, in Boston).
It has been a turning point because only one of the bugs which I considered release-critical was a JIT bug. The other issues were a bug in Gtk# which corrupted memory on x86, too and some endianess issues in PEAPI.
Now, it's time for fixing the more obscure bugs and then move on to do a bit of optimizations.

Mono vs JVM for the desktop
Both in the initial post from Havoc and in the replies some simplifications of the issues were made to keep the discussion focused and manageable, but when simplifications start blurring the data points, it's better to spell out the options we have available.
The issues to consider are technical, legal and strategic in nature and as Havoc points out each of the proposed changes to the way the Gnome desktop will be developed has stregnths and weaknesses in each of these areas. Let's see some of them.

Technical issues: as a platform, both Mono and the free software Java virtual machines are mostly on par maturity and stability-wise. gcj has currently probably the lead on portability (I guess, since it's based on gcc, but I have no hard data: anyone knows the status here?), while Mono currently has JIT ports to x86, sparc and ppc (though itanium, s390, amd64 and arm are in the works): things are moving fast on this issue in Mono, though, so I expect more ports to be completed by the end of the year.
From another point of view, Mono has better features to easily integrate with the existing codebase (P/Invoke and embeddability): I think this is important since the core libraries for Gnome are likely to continue to be written in C for the next few years.
Language independence has always been one of the strong points of Gnome and the architecture of the Mono virtual machine provides for a better foundation than the JVM: support for structures, function pointers, tail calls, etc, enable people to write compilers that target Mono and produce reasonably good code for a variety of languages. For example, while it is possible to run Java code in Mono (using IKVM) the opposite is not true. When you want to choose a platform that could be used as a base for Gnome, it's better to choose one which actually supports the use and the interoperation of different programming languages (either with compilers that target the virtual machine directly or with easy to use and implement bridges).
I won't go into the details of a C# vs Java programming language, but it looks like most people (who used to program in Java) found that C# fixed many of the issues they had with the language (though the new version of Java due out in the next few months implements some of them, too).

Legal issues: most of the legal issues are centered around the availability of the code under a free software license and on patents. The Mono code is available under the LGPL license (the runtime) and the X11 license (the class libraries). Some tools, like the C# compiler are GPL. GNU classpath is available as GPL+exception, so there doesn't seem to be any huge difference here, though Havoc cites legal issues with the use of GPL code and the proprietary JDK (to me this just means that the open source Java community needs to reimplement the relevant code as we have been doing in the mono project).
The patent issue is somewhat different: the major companies backing the CLR and C# standard agreed to RAND and royalty-free licensing of the patents needed to implement the standard. Libraries non included in the standard may have patent issues, though AFAIK no patent has been granted or claimed yet (this is a similar case as for the libraries we already use in Gnome development: no dying company claimed patents on them yet, but there is no guarantee that won't happen). This is a general issue of patents as has been pointed out multiple times, so it's not something specific to Mono. In fact there are likely several patents on Java and the JVM and no explicit license fortheir use has been granted.
So, for Mono, at least the core virtual machine and library is free from reasonable legal issues, while the JVM is not, as things are known today.
Of course, people see Microsoft as the devil and, as Havoc says, joining forces with Sun may give better results against Microsoft. I'll just note two things: both Sun and Microsoft are big companies with a variety of positions inside them. From one side, for example, Sun donated OpenOffice to the community (I consider it a contribution as big as the Linux kernel, the GNU tools and Gtk+/Gnome for the success of free software systems today) and from the other, they funded SCO, threatened JBoss etc. Microsoft, instead, is completely outside of our community; they, too, injected funds into SCO and they are competing against us. But at the same time they support the standardization of the CLR and C# as Sun never did for the JVM.
And rejecting a standard only because MS proposed it doesn't look like a rational move to me: I bet that today, 90% of the standard committees have MS representatives and a good chunk of the proposals come from them. Are we going to refuse to interoperate with the standard because of that? One of the reasons of the success of free software is that we tryed to implement standards whenever possible, to interoperate with other systems and to allow the migration from proprietary systems to open source ones.
So, in the end, it boils down to this: do you trust more someone who is ambivalent with our community and hasn't granted a license on its patents for our use, or someone who is outside, but did grant the patent license? Please don't answer that not all the possible patents on all the .Net technologies have been licensed, because I can easily reply that no patent license at all has been granted for the JVM:-) Also note that Microsoft is currently in a dominant position, but they may need to cave in on interoperability issues as a result of the antitrust case currently going to a (favorable for us) end in the EU.
I know some people will reply that they won't trust either Sun or Microsoft. What I think they are missing is that in the current patent situation any alternative technology has the same risks: it's not Sun or Microsoft the issue, but the patent system. Consider the issue this way: whichever technology we choose for the development of Gnome or of our applications and libraries, the patent risk is a constant. If instead you plot the patent and legal risk in a graph against the success of Gnome and free systems you'll find it's a steep, increasing line. If we want to be successfull in providing a free software mainstream platform we'll face patent and all sorts of legal threats (even if they are unsubstantiated, like the SCO case shows).
We'll be free of legal issues only if we become irrelevant.

Mono, Java or C++
I'll try to address some of the issues Havoc presented. Of course, I'm a Mono developer, so I'm biased, but hopefully people can see my arguments are more on the technical side than advocacy.

No rewrites please: this is a very important point: we can't just throw away the current code: we need incremental changes to not disrupt stability and compatibility. I'll just note that using Mono (and C#), interoperability with existing C code is much easier than with Java because of P/Invoke.

Calling managed code from C/C++: Havoc says it's hard, but Mono provides an easy to use interface to do that. Mono is designed to be embedded in existing applications, not just as a runtime for standalone completely managed programs. Also, it would be easy to create a shared library and header files to access managed methods seamlessly: they can be automatically generated thanks to the use of Reflection and the Mono embedding API.
I'm not sure a "simple native component system bridge" would solve the issues, mostly because simple systems are always found later to be incomplete, they get changed and become big, but with all the design warts needed to make a simple design work for not-so-simple constraints.
A minimal Mono system is currently about 2 MB on disk, but no effort yet has been put into reducing it (and I think it's entirely possible, we have been busy implementing features and leaving aside space optimizations). Of course, since the default build of the core assembly has lots of features, much of the reduction in size could be achieved by trimming features that other systems don't have:-). Even without trimming, most people will concour that 2 megabytes of disk space for a shared component is small enough in a desktop setting (and applications compiled to IL code are usually much smaller than comparable C apps anyway).

Community should decide: of course, I agree. Anything that is pushed down our throats by somebody else is not going to work for the free software and open source communities. The solution will need to be choosen because it actually solves issues the developers and the users see. Java had several years to try to attract developers from our community and it had some success in some niche areas (not for desktop applications, though). Mono has just started, but from the comments of the developers that actually used it to write new applications or port existing ones from C, it looks like we are on a good adoption path (even though we didn't release a 1.0 version yet, we are still working on debugging support and documentation is sparse).
Havoc fears the adoption of Mono or Java for the desktop would alienate people and cause forks. I don't think that will happen with Mono, because Gnome will continue to have a diversity of developers who'll prefer using the C libraries directly: Mono allows to keep and interoperate with existing code very easily and we want the migration to happen incrementally, so at first only end-user applications would be written in managed code, while the foundation would still be in C (at least, enough of the foundation to have people happyly writing their own apps in c or with the existing bindings). At that point, when a managed execution environment has proven itself to both developers and users (hopefully) we could start discussing about using it for the foundation, too, if that makes sense. I think Mono is positioned better here to allow this incremental shift of both development and espectations towards a managed runtime.

Problems with a .Net clone: Havoc claims that MS controls the platform because, even if the core is unencumbered, some assemblies are tied to MS technologies and there is non standards body or community momentum to build alternative solutions for a complete platform. Well, considering that until a couple of months ago there were 5 people developing mono, we have achieved a lot, not only in the implementation of the runtime, but also, thanks to the large community contributions, in other areas, such as Gtk#, database connectivity, security. There is already a large body of code that is completely independent from the MS stuff. Besides, I don't see anywhere an assurance that Java and all the libraries included with it are unencombered, so this is not an argument that applies to mono only. Sun likely has several Java and JVM-related patents, so a free software implementation of Java is no less a possible target for patent lawsuits than Mono (and probbaly more so, since the companies that submitted the CLR and C# ECMA standard agreed to license the patents needed to implement the technology on a RAND basis and additionaly royalty-free: AFAIK Sun didn't do any of that). Sun went to court to defend Java's purity: will it also go after free software implementations of it if they want to innovate and diverge from the spec?
Anyway, as an example: MS have their own Java-on-.Net compiler and runtime system. Mono didn't implement that and uses instead the excellent IKVM.NET free software JVM implementation to run Java code.

Innovation: It should also be noted that we (as in Mono developers) are very open to input from the community about changes to the runtime and base class libraries that could be needed to better support new features, independently from what the MS runtime does (the topic came up, for example, about supporting dynamic languages on Mono). Of course, the programs wouldn't run unchanged on the MS runtime, but we can't have it both ways: if we have to innovate we'll be the first to be able to run such programs and maybe later MS will adopt the same technology:-) The only consideration is just this: not breaking gratuitously the compatibility, because that always has its cost. Most of these changes can simply happen in the same way gcc introduced extensions to the C language: first just gcc users could use them, later some were adopted by C99.
Of course, since we've been busy implementing the base platform, we didn't have much time to propose and implement innovative changes, but expect changes in the next few months (hopefully also from the community involvement). There is no need for us to just wait for MS to implement something and then to re-implement it as free software. We welcome your contributions in enhancing Mono in ways that can benefit the free software community. We should also consider that ECMA standardizes existing practices (hey, where did I hear "Show me the code!" last time? Yep, in the free software community): we're willing to assist people in implementing prototypes of innovative changes in the CLR using Mono as a base and submit them to ECMA for consideration.

Compatibility with the devil: We (as in Ximian/Novell and AFAIK also many mono contributors) will try to keep up with changes done by MS to the platform mostly because it enables people to move from the MS platformto a free software platform more easily: in the next few years developers on MS will be forced to develop for the managed platform and we'd like to offer them an alternative to escape to the free world. I think this will be a good thing for the free software community, but I can see other free software developers not caring. I can live with that:-) Just don't use it as an argument against Mono:-)
Havoc notes that ECMA defines only a subset of the .Net platform and that it maybe too controlled by MS. Even if MS would try to undermine the changes we'd submit to ECMA for consideration, this doesn't mean we can't have our own, community-driven standards effort for enhancements to Mono and for the development of independentclass libraries. Let's discuss how can we best build a standardization effort.
In Mono there are two different stacks of code: the MS-compatible ones and the free software/community one. See the picture. Free software people can just ignore the MS-compatible bits and let the developers who want to port their code from Windows to Unix systems worry about that.

Spelling out the options: I'll just note here that one of the options is simply to use Mono and allow people who prefer the Java langauge to use IKVM as a bridge: this may be the best of both worlds. A free software foundation that doesn't exclude (choosing Java would exclude C#, since there is no way to run C# code in Java or any of the other languages that target, or will target, the CLR).
Havoc claims that "Anything which is defined as a .Net clone appears to not be viable", because it would speedup the adoption of MS technology. I think that for people who develop for the MS platforms what Gnome will choose is largely irrelevant. Besides, Mono enabled people to jump from the MS platforms to the free software and open source community. So far it has been a win for us, judging by the contributions we had to Mono from formerly MS-only-oriented developers.
Anyway, the solution to that is simple: just don't call Mono a .Net clone:-) Mono is open to input from the community for enhancements: if differentiating mono gives benefits to the community adoption and to Mono as a development platform, I'm all for it (just let's not break compatibility gratuitously).
Havoc says: the open source Java subset is something we could start using today,that nobody has any fundamental reason to reject. Well, using Java as the foundation has issues, the first of which is that it doesn't easily allow incremental migration to a new codebase. It also excludes C#, since no JVM can run C# code. I think starting out with the Mono core would be better, both for interoperability with exisiting code and with Java, too (using IKVM).

DV: the following javascript snippet may be of help dealing with spam to mailman-managed lists. It needs spamassasin to mark messages first, but it can be easily adjusted to look for different patterns in the header or in the body. The spam level that triggers automatic rejection can be adjusted as well: the current value works well enough for me.

javascript:for(i=0;i<document.forms[0].length;i++){
        if (document.forms[0][i].value=='3') {
                var name = "headers-" + document.forms[0][i].name;
                var headers = document.forms[0][name].value;
                if (headers.indexOf ("X-Spam-Level: ********") != -1) {
                        void(document.forms[0][i].checked=true)
                }
        }
}

fejj: you got the SIGBUS because of an unaligned store: a pointer is stored to 0xa8c52 (aligned to 16 bits instead of 32). A backtrace would be useful to see where the data pointer is coming from, but I guess the mailing list is a better forum for bug reports than advogato:-).

• lupus certified esr as Despicable person

Yesterday I got the ADSL connection at home: they were fast, since I requested the connection on Sunday. It looks good so far: I get 60+ KB/s from the Debian mirror...
I still need to fix the ethernet cable on the wall since currently it's spread all over the living room:-)
Anyway, in a couple of days I should be back on speed with my free software projects and my email backlog.

The Perl compiler tools are very powerful: I wrote a backend to implement xgettext for perl programs (needless to say using regular expressions to pick strings is not going to work very well:-).

I also wrote a module that allows the use of the gettext related functions from perl. Using closures it makes for a very easy interface also for modules. All the code that's needed is:

use gettext 'package';

and then marking the strings with __:

print __"Hello, world\!n";

I'll post it to CPAN for comments.

Ok, after such a long time without a diary entry it's hard to come up with an excuse that's not lame, so I'll skip that part:-).

Last events at my company have been quite interesting and enlightening: it looks like things are shaping up now and the future seems brighter.

That same events allowed me (well, in the time between the various company meetings) to hack on a few little things and ideas.

One result is dh-make-perl a program that creates a debian package out of a perl module. It can also automagically retreive the module from CPAN. The use is as simple as:

dh-make-perl --build --install --cpan 'Some::perl::module'

You can find it in the woody distribution.

I used it to try to impress ettore and make him switch to Debian from that other distro: still no luck, but I'm confident and time is on my side ;-).