Since I restored (actually rewrote) the machanism of nightly (or daily?) snapshots of SableVM I've been thinking of also performing automated testing. We have our own small and highly specialized test suite for testing SableVM's bytecode execution engine, but this one is mostly useful for porting SableVM to new architectures (especially for porting the (fastest) inlined engine). Once this test suite runs well, there's little chance we can hit any VM-related troubles later, during usage. I had little time lately (end of fall semester) so I only tried running Mauve twice and either it was just hanging after a few tests or all the ran tests were failing. Not getting into the details, on the 3rd try... heh. You can judge by yourself how natural you think such a "good" call to run Mauve is:
export JAVA=/usr/bin/java-sablevm; export JAVAC=/usr/bin/jikes-sablevm; export COMPILER=$JAVAC; export RUNTIME=$JAVA; ./configure --with-mailhost=localhost && make && ./batch_run &>./output
But then I was able to finally get some results. I did the test with the latest Mauve, but I used the debian package of SableVM 1.1.6-6 which is to be released with Sarge. This version is fairly stable and includes many bugfixes that made it into upstream 1.1.7 later, but its classpath is not the newest. Yet, I've heard that with the latest classpath about 250 tests are expected to fail, which I got about 350 failing. Given that mauve has some 10000 tests (IIRC), this is quite a good result. I need to setup automated testing when I have some free time (December?)
Some fun stuff - repeated due to high "hit rate" :-)
I've noticed that SableVM Free Java Virtual Machine Hall Of Fame has been very popular (webalizer was helpful in realizing that fact :-) so I am just reposting the link. I am still looking for new links to put there, so don't hesitate and just add them to the Wiki!
I think I like the idea of doing monthly releases. We haven't been able to really do this until now, because of the lack of automated testing. That is, it'd be easy to just release whatever we have mostly working in the development branch, but we would risk shipping something buggy. So we've always had to manually run certain tests - which always takes time so you don't want to do that too often.
But as soon as we get some automated testing we should be able for ex. to bind releases of SableVM with snapshot releases of GNU Classpath, which, according to what I've heard from Mark, should now be monthly. That would fit quite well! (yes, I know that now you can run SableVM Java Virtual Machine out-of-the-box with GNU Classpath CVS, but still - the releases are more useful to regular users).
PS: If everything is fine, we should have an interesting and useful Christmas gift for the whole Free Java community. Hopefully we'll make it on time ;-)