Older blog entries for rlougher (starting at number 19)

16 Feb 2011 (updated 16 Feb 2011 at 04:14 UTC) »

OpenJDK/JamVM Git repository

JamVM's got a shiny new Git repository, which contains the port to the OpenJDK class-library. Details are here: http://developer.berlios.de/git/?group_id=6545

The repository can be checked out anonymously with:

git clone git://git.berlios.de/jamvm
JamVM 1.6.0 will be released off of this in the near future. JamVM 1.6.0 will be a combined release, supporting both GNU Classpath and OpenJDK class-libraries, with GNU Classpath support being built by default. I still need to run some tests to make sure that the refactored codebase hasn't introduced any regressions w.r.t. GNU Classpath and JamVM 1.5.4.

So what can be done with the OpenJDK port? As discussed in my FOSDEM talk, it is ready for a first release. There's stuff which hasn't been implemented, but it runs everything I've tested it with (jedit, eclipse, derby, etc.).

Some words on running the port. JamVM is not yet integrated into the IcedTea build process (although it supports the same --with-java-runtime-library switch as Cacao). Instead, the easiest way to test the port is to build JamVM, and copy the libjvm.so file into an existing IcedTea/OpenJDK installation.

After cloning the git repository, do:
./autogen.sh --with-java-runtime-library=openjdk
This will generate the autoconf/automake files and configure JamVM to build support for OpenJDK.

Then do make, make install as usual. This will put libjvm.so into /usr/local/jamvm/lib.

This can then be copied onto an existing IcedTea installation (or a copy of one), e.g. on x86_64 (as root):
cd /usr/lib/jvm
cp -r java-6-openjdk jamvm-openjdk
cp /usr/local/jamvm/lib/libjvm.so jamvm-openjdk/jre/lib/amd64/server
You can then run it by running the normal java wrapper, e.g.:
  /usr/lib/jvm/jamvm-openjdk/jre/bin/java -version

java version "1.6.0_20"
OpenJDK Runtime Environment (IcedTea6 1.9.5) (6b20-1.9.5-0ubuntu1)
JamVM (build 1.6.0-devel, inline-threaded interpreter)
(the inline-threaded interpreter is the other name for the code-copying JIT)

Syndicated 2011-02-16 01:12:00 (Updated 2011-02-16 03:31:20) from Robert Lougher

Debian Linux on cheap MIPS mini netbook

Computing-wise, I've taken a break from the JamVM/OpenJDK port for a couple of days while I play with my latest toy : a cheap mini-netbook based on a Chinese MIPS clone. It's branded CnMbook, but it's available (or was) under dozens of names.

Yes, it's been available for a while. I first investigated it over a year ago as I wanted a MIPS machine on which I could do the port of the code-copying JIT (I did the MIPS port of JamVM back in 2007, and I've not touched it since). But it was 170 pounds, which was too much for the tiny spec, and I got an EeePC instead.

However, I still don't like Intel, even though it finally became my main architecture in 2008 (albeit AMD). And I like it even less as a mobile processor, as it's the last area where it doesn't dominate.

So what lead to this sudden surge of interest? I've been waiting for years for the fabled ARM-based netbooks, and a couple of weeks ago I saw a cheap WinCE mini-netbook in a local discount shop which I was passing. My brother (the author of SquashFS) thought it shouldn't be too difficult to get Linux working on it. This lead to a weekend long investigation, which showed up some startling results.

The mini-netbooks appear to be based on one of two ARM9-based SoCs (so these still aren't the low-cost Cortex A8 ones I've been waiting for). Either the Anyka AK7802 or Via's WonderMedia VT8505 SoC. The amazing thing? There's no publicly available Linux kernel (with source) for any of them. There is a binary-only Android kernel for the VT8505 but no public source. For the AK7802 there's nothing, and as Anyka have not made any SoC documentation publicly available it's likely that there never will be.

In contrast, Ingenic, the maker of the MIPS clone in the CnMbook have made available the full source to their modified kernel, u-boot bootloader, rootfs and applications. It came with CELinux, and an ancient 2.4.20 kernel. But it didn't take my brother long to get a modern-ish 2.6.24 kernel running on it (he's working on 2.6.31), and with Debian MIPS Linux it makes a nice little portable development system. Compiling is rather slow, but vim runs well on console, which is all you need!

The cost? 65 quid off ebay for an ex-display model as you can't buy them anymore, the ARM9 WinCE machines having completely replaced them.

I'll next post my experience of running JamVM/OpenJDK on it :)

Syndicated 2010-08-03 22:15:00 (Updated 2010-08-03 23:47:35) from Robert Lougher

... a thousand words (JamVM/OpenJDK update 2)

Firstly apologies to the people who commented on my first progress update (18th May). I'd hoped to do a blog update way before now, but I've had a lot less time to work on JamVM/OpenJDK port than I expected...

Xerces Ranby asked:

Does JamVM still produce those quick and fast startup times when using the OpenJDK class libraries compared to the fast startup times obtainable when using GNU Classpath classes?
[best of 3 runs]
rob@traken:~/JAM/tests$ time /usr/lib/jvm/jamvm-openjdk/jre/bin/java -showversion hello
java version "1.6.0_0"
OpenJDK Runtime Environment (IcedTea6 1.6.1) (6b16-1.6.1-3ubuntu1)
JamVM (build 1.5.5-openjdk, inline-threaded interpreter)

Hello World!

real 0m0.046s
user 0m0.030s
sys 0m0.000s

rob@traken:~/JAM/tests$ time jamvm -showversion hello
java version "1.5.0"
JamVM version 1.5.5-devel
Copyright (C) 2003-2010 Robert Lougher
...
Build information:
Execution Engine: inline-threaded interpreter
Compiled with: gcc 4.5.0 20100211 (experimental)
Boot Library Path: /usr/local/classpath/lib/classpath
Boot Class Path: /usr/local/jamvm/share/jamvm/classes.zip:/usr/local/classpath/share/classpath/glibj.zip

Hello World!

real 0m0.048s
user 0m0.030s
sys 0m0.020s

gnu_andrew asked:
Will JamVM still support GNU Classpath, as CACAO does?
Yes, most definitely. I still consider GNU Classpath as JamVM's main class-library as it's where my chief loyalty lies. On more practical grounds, even after the OpenJDK port is functionally complete it will still be a long time before it is as tested as JamVM/GNU Classpath. FWIW, many embedded systems seem to be quite happy with GNU Classpath. GNU Classpath is considerably smaller "out of the box" and much easier to build...

As far as development is concerned, I've taken a different approach to Cacao. Cacao implements the class-library differences within the VM-specific code using #ifdefs. While there's nothing wrong with that, I personally think it makes the code harder to read, and it's harder to get an overview of the changes.

Instead, I've tried to abstract the differences into a classlib interface. At times this has taken some thought and quite a lot of code re-arranging. If anything it's made the code cleaner, as a lot of the messier details are hidden (in general, I'm not a fan of information hiding, but removal of some of the VMFoo details makes the intent clearer).

Having said that, the classlib interface is mostly driven by the differences between GNU Classpath and OpenJDK as I find them. I'd like to think the interface is reasonably generic, but it will probably need changing if another class-library came along...

Currently the classlib interface has 40 functions, the gnuclasspath directory has 8 files, totalling 2552 LOC, and openjdk 10 files totalling 3491 LOC.

Christian Thalinger (hello, twisti!), linuxhippy (hello, Clemens!), Michael Starzinger (hello, Michi!), Stefan Ring:

Yeah, it's taken a long time, and lots of prevaricating. It's not been quite as boring and tedious as I expected; some of it I've actually enjoyed :) Debian might even re-instate JamVM (sticking pins in the Debian T-shirt I bought at FOSDEM, while looking for the Fedora 13 CDROM).

Syndicated 2010-07-26 23:19:00 (Updated 2010-07-27 01:30:21) from Robert Lougher

18 May 2010 (updated 18 May 2010 at 01:07 UTC) »

JamVM and OpenJDK - Progess Update

Porting JamVM to use the OpenJDK/IcedTea class-library has been on my TODO list for quite a while. It appears on my FOSDEM slides for 2008 and again in 2009. It doesn't this year, as I didn't do a talk. Maybe that's why I've finally got round to doing it. I told Twisti I'd get "Hello World" running in 3 months. I've missed by just over a week, mainly because I only started 2 months ago.

root@traken:/usr/local/jamvm/bin# java -Xbootclasspath:/usr/lib/jvm/java-6-openjdk/jre/lib/rt.jar -Dsun.boot.library.path=/usr/lib/jvm/java-6-openjdk/jre/lib/amd64 -showversion hello
java version "1.6.0_0"
OpenJDK Runtime Environment (IcedTea6 1.6.1) (6b16-1.6.1-3ubuntu1)
JamVM (build 1.5.5-openjdk, inline-threaded interpreter)

Hello World!
root@traken:/usr/local/jamvm/bin#

There's still quite a lot to do, including finishing off support for reflection and sun.misc.Unsafe but threading, exception handling, library handling, class-loading and the VM bootstrap sequence should be done.

Syndicated 2010-05-17 23:36:00 (Updated 2010-05-18 00:14:21) from Robert Lougher

Filesystem Collaboration

My twin brother is the author of SquashFS. A very influential Linux filesystem used on practically all Live CDs. He reads the Linux Magazine interview with Valerie Aurora who says:

hoping that if the existing Linux file systems developers could just talk to each other, we could come up with a way to improve funding for Linux file systems development. I don’t know that it helped, but if it did, I consider this to be my most important contribution to Linux file systems.

So decides to link up via LinkedIn, and gets refused...

Syndicated 2009-07-17 23:41:00 (Updated 2009-07-17 23:54:49) from Robert Lougher

12 May 2009 (updated 12 May 2009 at 15:11 UTC) »

Netifera uses JamVM as remote probe

Netifera is a very interesting 100% Java-based Open Source network security platform. The next version will include a probe, which is a "deployable software agent that makes it possible to run all netifera platform tools remotely".

The probe uses JamVM and a cut-down version of GNU Classpath. This is downloaded onto the remote site to run the netifera tools. My thanks to Claudio Castiglia (of Netifera) for sending me the following link to a webcast showing the probe in action:

http://netifera.com/video/netifera_java_virtual_machine_as_shellcode


JamVM and GNU Classpath are mentioned starting 14:54 into the webcast in the discussion of the probe architecture.

Syndicated 2009-05-12 01:04:00 (Updated 2009-05-12 14:11:42) from Robert Lougher

JamVM on Beagle Board

I've been intending to get a Beagle Board for quite a while now, to replace a Neo1973 as the development platform for JamVM on ARM. The Neo has an ARM920T core, and I've been particularly wanting to test JamVMs code-copying JIT on a more modern implementation (the Beagle Board has a Cortex A8). But I've been waiting for the Rev C board, as this has working EHCI USB host, and 256MB RAM (previous revisions had 128MB). The board arrived 2 weeks ago, but I haven't had much time to set it up until now...

So far, the results are quite encouraging. SciMark 2.0 shows a 55% speed improvement:



While jBYTEmark shows an 82% improvement on Integer tests:



The option -Xcodestats prints out the size of the JIT code-cache when the VM exits. This shows 77K was used after running SciMark 2.0, and 178K after jBYTEmark. This is using the code-profiling introduced in JamVM 1.5.2.

Syndicated 2009-05-11 21:55:00 (Updated 2009-05-12 00:53:32) from Robert Lougher

17 Mar 2009 (updated 17 Mar 2009 at 11:17 UTC) »

JRuby 1.2.0RC2

I read with interest Jeroen's recent post on his blog about his experiences running JRuby 1.2.0RC2 on IKVM. I know I don't run enough "real-world" tests on JamVM, so I finally got around to trying to run it on JamVM over the weekend. Then wished I hadn't. I secretly hoped it would "just work", but it almost immediately segv-ed. This was the first of 5 problems.

The segv was relatively easy to find and fix. A regression introduced in JamVM 1.5.1, when I added unloader objects to unload JNI libraries after the classloader which loaded them is garbage-collected. This was itself a fix for library unloading, which used to be done within GC, but broke if the JNI_OnUnload function called back into Java.

Basically, I didn't take into account libraries which have a JNI_OnUnload function being loaded by the boot classloader (the NULL loader). This is pointless, as the boot classloader is never collected, and therefore no library loaded by it can be unloaded. However, the fix was simple - just ignore them.

The next problem was with MemoryManagerMXBean. I spent some time implementing native support for ThreadMXBean in JamVM 1.4.4 as part of the thread re-work, but never got round to implementing the full set, as nothing much seemed to use them. For now, a simple implementation which just returns "no memory managers" appears to be sufficient.

After that there was a problem with annotations, where an AnnotationTypeMismatchException was being thrown. This took some time to track down because I had to remember how annotations worked! It ended up being a mismatch between an annotation array value and the method return type. When parsing the annotation, the array values can be any one of a number of types, so in JamVM an Object array is created (when the array is created, the elements haven't yet been parsed so the type isn't known). But the method return value is the specific type, in this case String[]. Luckily, a similar problem has been found with the implementation in gcj, so I was able to adapt the fix into JamVMs version of sun.reflect.annotation.AnnotationInvocationHandler.

Problem number 4 was with VMClassLoader.getPackage(). The default GNU Classpath implementation relies on a META-INF/INDEX.LIST file existing in the first Jar in the bootclasspath. JRuby uses Constantine, which uses the package name to load an appropriate constant class. As the Constantine Jar is added to the bootclasspath, even if the INDEX.LIST existed, it wouldn't have any package information for it. A quick and dirty implementation of VMClassLoader.getBootPackages() which doesn't need INDEX.LIST fixed this.

Finally, there was a problem with Class.getSimpleName(). The simple name is appended to the package name to locate the constant class. However, the GNU Classpath implementation of getSimpleName (which delegates to VMClass) is broken. Again, I took the fix from gcj.

After all this, jRuby runs!

rob@dougal:~$ jruby hello.rb
Hello World!

The next thing to try was jirb (interactive shell). For some reason, the default prompt doesn't work (nothing is shown, it may be related to sun.misc.Signal, as jirb complains about an unsupported trap). However, the simple prompt does.
rob@dougal:~$ jirb --prompt simple
trap not supported or not allowed by this VM
>> include_class java.lang.System
include_class java.lang.System
=> Java::JavaLang::System
>> System.getProperty("java.vm.name")
System.getProperty("java.vm.name")
=> "JamVM"
>> System.getProperty("java.vendor")
System.getProperty("java.vendor")
=> "GNU Classpath"
>> quit
quit

Syndicated 2009-03-17 01:18:00 (Updated 2009-03-17 10:42:55) from Robert Lougher

Benchmarks

A couple of people after my talk at FOSDEM asked me where the benchmarks were. I didn't have any partly because I didn't think there was time in the talk, partly because I didn't know what to benchmark against, and mostly because I hadn't had time to do any.

So, I've finally done some. I've benchmarked against the JDK template interpreter, because of the interest due to the recent work of Shark and Zero. The benchmarks are from SPECjvm2008. Not all of these run with GNU Classpath, and from the rest, I've selected benchmarks which should hopefully not be influenced by the different class libraries.
Note, these benchmarks are not compliant, and I am in no way publishing these results. For the record, the benchmarks were ran on an Athlon 64x2 3800+.

Syndicated 2009-03-02 00:41:00 (Updated 2009-03-02 02:47:54) from Robert Lougher

10 older entries...

New Advogato Features

New HTML Parser: The long-awaited libxml2 based HTML parser code is live. It needs further work but already handles most markup better than the original parser.

Keep up with the latest Advogato features by reading the Advogato status blog.

If you're a C programmer with some spare time, take a look at the mod_virgule project page and help us with one of the tasks on the ToDo list!