Advogato blog for audriusa

23 Mar 2008

Sun, 23 Mar 2008 21:31:10 GMT

"GNU hackers flatten Sun's professionals with they java.util.BitSet" - sounds not so bad. Some simple comparisons show that our BitSet runs roughly 24 % times faster than Sun's 1.6.0 implementation. Sun takes back on HashSet, however - this one is slower in GNU Classpath, more or less by the same percent. See the posted Advogato article for details.

I am not sure why our BitSet is faster. Deciding from OpenJDK, both implementation use the array of long's to keep the data. From the other side, the overall code is so different that it is difficult to say, who makes the benefit. The history of GNU BitSet spans over seven years (1998 - 2005) and have seen many contributions. The main authors of this class seem Jochen Hoenicke, Tom Tromey and Eric Blake.

I have checked near all java.util classes but the remaining differences seem for me too small to be considered seriously. I am, however, happy to discover that GNU Classpath is very far indeed from being universally worse than OpenJDK at any single line of code. The detailed 'class versus class' check may discover more interesting differences.

22 Mar 2008

Sat, 22 Mar 2008 15:08:42 GMT

Today I started to do something that I am sure it was needed long time ago already. I took the latest GNU Classpath distribution and started to rip some selected interesting package, java.util, apart. One time I will check properly if it is worse or maybe same or maybe better the one we have got from Sun.

GNU Classpath has been written over long time by numerous developers. In addition, it usually runs with the different java virtual machine than the Sun's code. In that way, no honest comparison is possible between any units that are smaller than all jre + all rtl together.

So, that I have done so far I moved the chosen set of GNU Classpath java.util classes into some transient package where they do not conflict with java.util from Sun. Some code editing was needed to build everything but in general it was trivial to build it that way. Now I have the two java.util's on the same virtual machine! It is time to try some performance comparison. We will just write some simple tests for that. Stay tuned.

1 Mar 2008

Sat, 1 Mar 2008 10:25:13 GMT

FOSDEM 2008
Unlike two years ago, this time I have been a "completely different person" in FOSDEM: arrived by plane, used hotel and also brought two young developers I supervise - to demonstrate them how the FOSS looks like from the closer distance. Even a very simple things are impressive for some people: that the conference was organized by the university, that it was really large, that the level of the projects is indeed very high and that nobody does not even think to talk about things like cracking iPhones. It is important to resist various FUD in time. This goal seems reached and I am satisfied with the result.
Apart java talks, one of the most interesting things I have seen might be OpenSolaris. While I needed to stuff an extra hard drive into my box just to install it properly, I did - runs fine, scaring the older staff from the oil industry like a resurected ghost: Solaris??? Its dead! It is over! Well, there are many other interesting things - the summarized report was 16 pages long. Really a nice weekend.

27 Nov 2007

Tue, 27 Nov 2007 20:22:50 GMT

... this is to ask a few minutes of your time to help my research group...

Today I haver received one more questionnaire related to the "FOSS research". It is already a second one this month, and as much as I remember already a fifth I have completed. Some wave of "fundamental scientific analysis" on FOSS seems spreading around the world. Why are you programming this? Are the companies involved? Do you feel the owner of your project under GPL? Would you program if you know it would be illegal? How the important development decisions are made inside your project?

Somewhat it was considerably less attention even a year ago. While FOSS developers likely have never be treated as a bunch of hackers about that there is "nothing interesting to know", it seems an achievement that more people accept FOSS development as a kind of process which needs detailed study and understanding. Something similar to the mountain formation.

Hope they will not fish out any weak places in our movement with these surveys. If they potentially could, it is likely time to think about the shared policies how to answer...

9 Nov 2007

Fri, 9 Nov 2007 20:47:56 GMT

libgcj in C-based parallel clusters?

Today I was setting the MPI cluster. A tcl-based expect script is required to arrange the MPI connections. Expect was not present on my SuSe machines by default, but I found it between optional packages. Still, all that this default installation was doing is it was silently hanging with MPI setup script. Hence our team needed to build the more recent expect distribution from the source, remembering all fun of --configure--with (the default ./configure does not find the tcl configuration script on 64 bit platforms).

We had problems with firewalls, also. MPI uses same approach as the old CORBA implementations did: just opens multiple random ports wherever they like. However our 'cluster' is not just for distributed computing - people use them heavily for various purposes, connecting via ssh -X. Finally we decided to use combined mac + ip address filtering with iptables.

Ok, the MPI cluster is now running with full Fortran and C++ support. Likely we will enjoy it, but how to build a bridge back to java? Some of our java programs already have complex GUIs, others are heavily JSP-based - the prospect to rewrite this stuff in C - even parallel C - does not seem attractive. I is now nice to have a java runtime library that connects with C easily, without the need to load and start all java virtual machine as a separate process. All we need is java-style serialization, RMI and maybe RMI-IIOP. It is far less than GNU Classpath is capable to do. Likely it makes a lot of sense to try our old staying-alive libgcj here ...

27 Oct 2007

Sat, 27 Oct 2007 16:48:41 GMT

Yesterday I was setting up CORBA via SSL on JacORB and was surprised that despite seemingly good support this task can still be such a headache. We needed the single-port SSL connection that would be secure from beginning (no naming service on an unprotected port) and would not require to pass the lenghty server IOR address via some other channel (the example in JacORB uses a shared file system to pass the IOR and in addition is broken in 2.3.0 release). The corbaloc:ssliop address notation still seemed not supported, so the only hack around we were able to find was to patch the pre-generated IOR for the local host with the separately given IP address and SSL port. Thank goodness JacORB has the internal undocumented routines to parse and modify IOR's, same as the GNU Classpath does. It is really sad that these IOR manipulation routines are in the private space and may change dramatically between releases. Likely having the useable naming service via SSL connection without passing of the 2K IOR that must be known in advance should be very easy and trivial to do.

I know that people more talk about the web services in these days, but our "messages" are many megabytes in size even when they are binary. It is really great that I have a lot of things to remember from the GNU Classpath CORBA project - otherwise it would likely take days to figure everything out. Indeed, the company managers should look for the Free software developers to do they tasks ...

21 Oct 2007

Sun, 21 Oct 2007 16:37:24 GMT

GNU Classpath 0.96 - "Staying alive"

Again, a new version of GNU Classpath, our old Free java runtime library, has been released - GNU Classpath 0.96. Apart version numbers, many releases also have smart names: A La Mort Subite, All for One, One for All, Dreamland ... But for everybody who is more or less aware about the current history of java, the recent name of 0.96 sounds more aggressive than any previous. Staying alive. Too pretty to die.

Who have changed since the times of 0.93, when multiple GNU Classpath hackers, myself including, were demonstrating fully functional GNU Classpath Swing for the astonished public in a number of software conferences? How happened that the nine year old project with near two millions lines of code feels proud just surviving? Well, we, surely know: Sun have released the majority of its own java runtime library under GPL. Soon it will not be necessary to use and develop the GNU Classpath code just because it is Free when Sun's code is not. Unless, of course, other reasons would appear.

Somebody who is not aware about the current state of the GNU Classpath may say: that is fine, just merge the Sun's code in! However while Classpath continues getting multiple patches per day, no significant merging activity have ever been seen. The reason is simple: GNU Classpath implementation is too complete. There were several useless, non functional stubs in the past (RMI-IIOP implementation, for instance). However it is long time since they are replaced by the fully working code. There are no parts that are just non-functional, clearly deserving to be ripped out, stuffing the Sun's code instead. Or should we believe that any single line of code from Sun is doubtlessly better? Written by professionals? That do you know about the GNU Classpath team, if you think so?

Yes, there are tasks where the current Sun's 1.5 implementation (with some pretty proprietary stuff inside remaining, by the way) performs faster, but does this mean that GNU Classpath should now be abandoned? The programming user who does not want to spend any time comparing the sources of these two implementations would likely just pick the variant that is more efficient today. But it is possible to imagine projects that dig deeper than just using java. Now, then the Free Sun's implementation starts emerging, the major value of GNU Classpath may be in that it implementation is different.

From one side, GNU Classpath implements the same official API. But from the other side, no GNU Classpath hacker has ever been allowed to look into Sun's code while writing the alternative implementation. This means, that many or even majority of the algorithms in Sun's and GNU's implementations are realized differently. This opens a really exiting prospects to compare the solutions, picked by various developers when implementing exactly the same framework. From the first sight, this comparison may look very difficult and work-intensive to do. However we should not forget that (differently from GNOME and KDE, for instance) we speak about the system where the modules of the alternative implementations are aligned against each other. Packages consist of classes, classes consist of methods and these methods must have the exactly the same functionality implemented. Hence it is surely possible to compare package versus package, and in many cases it should be possible to compare even class against class. It may well be - and I personally am sure in this - that some methods, classes or even packages exist where GNU Classpath implementation may actually be better. Yes, that is not funny. During development I did a lot of side-by-side comparisons at least for CORBA and HTML parser (both were later continued by my friends) and have reasons to suspect that Sun might not be an absolute winner in all possible test cases. Also, it may be other parts of code where the better implementation can be written after comparing the two existing ones and learning from mistakes of both sides. It is not frequent to have opportunity to compare the two different implementations seriously, package versus package or may be even class versus class.

Hence it may well be that the best Free java runtime library is yet to be written, uniting the best solutions from Sun's, GNU Classpath and Apache Harmony implementations. Such project can be an independent initiative, but it also may start as a branch (or even head) of the GNU Classpath project. Hence a serious hacker can find a new ways to the future of the GNU Classpath. Ways that do not look just like a gradual decay. Ways that lead forward though staying alive.

6 Oct 2007

Sat, 6 Oct 2007 19:11:07 GMT

Linux in the USB stick - direct installation

Recently I checked the old idea to install Linux into USB stick directly, as into any other hard drive. This would be very simple and straightforward approach: just plug in USB drive, boot from the installation CD and expect to the /dev/sda (or similar) between devices during installation. Then direct the installation into that drive and tell the installer to leave other disks alone. The approach is not directly dependent from the distribution and looked very attractive for beginners.

Debian have found the attached USB drive and fully completed the installation. In only asked me a couple of times if I really do not want a swap. I said I do not (how reasonable is to have a swap partition even on 4 Gb USB drive?). GRUB, however, was not configured correctly: in device map and menu.lst the pen drive (/dev/sda) appeared as hd2 and the flash did not even think to boot. However all I needed was to correct the boot device name to hd0 in menu.lst. The attempt to get rid of hard-coded /dev/sda name in /etc/fstab (using the LABEL approach) was also fully successful.

OpenSUSE that we use at work was even smarter in the beginning: GRUB was configured correctly and it was possible to boot from the flash immediately. Just fstab needed corrections to get rid of hard coded dependencies, again. Also, both Debian and OpenSUSE installers configured the graphic and network settings along the lines of the hardware that they have found on that particular machine - so if the machines from where you will need to boot are different enough, it is necessary to switch into something like VESA manually afterwards (xdebconfigurator seems a magic curse for Debian).

Still, I find this experiment interesting. It seems that with very little additional effort various Linux distributions could support USB stick a rather specific (generic drivers must be chosen) but still 'just one more hard drive'. I know that some distros have the 'install into USB stick' item on a user friendly level, but it does not necessary work well either. While I do needed to discover and fix several nonsenses myself, there are a lot of web sites where installation into USB drive is described as something horribly more complicated (see Debian instructions, for instance).