Older blog entries for gary (starting at number 249)

JNI wrapper compilation

I now have a version of Shark with a basic implementation JNI wrapper compilation. Sadly I can’t say if it’s faster or not yet because it’s totally unstable!

The problem is this. When HotSpot wishes to compile a normal (interpreted) method, the thread initiating the compile simply adds it to a queue and carries on doing whatever it was it was doing. A separate thread, the compiler thread, loops over this queue, compiling methods one at a time. This means there’s only ever the one thread making LLVM calls, and everything is rosy.

When HotSpot wishes to compile a native (JNI) method, the thread initiating the compile bypasses the queue and does it immediately. It acquires a lock, the adapter hander library lock, so there’s only every one native method being compiled at once, but in the meantime the compiler thread is in all likelihood busy compiling some normal method or another, so there’s two threads making LLVM calls. LLVM doesn’t like this, not LLVM prior to 2.6 at any rate, and even then not without being written with a separate LLVMContext for each thread.

The obvious fix for this is for Shark to acquire a lock before compiling either a normal or a native method, ensuring that only one thread is calling into LLVM at once. This doesn’t work, however, as the compiler thread runs _thread_in_native. The benefit of this is that the compiler thread does not have to halt for safepoints (and the rest of the VM doesn’t have to wait for the compiler thread to halt) but the drawback of this is that threads running _thread_in_native may not own locks. You can’t make the compiler thread run other than _thread_in_native, not without losing the large chunk of the server compiler that Shark shares, and you can’t hack a lock in there anyway (by using a pthread mutex, say, rather than a HotSpot lock) because it’ll deadlock the first time a safepoint occurs when the compiler thread holds the lock (compiling a normal method) and a Java thread is blocking trying to take it (to compile a native one).

I’ve been circling around this issue for a couple of days now, but it looks like the only solution is to require LLVM 2.6 and rearrange everything to use separate contexts.

Syndicated 2009-10-28 10:57:11 from gbenson.net

Zero, now available upstream

So the two halves of Zero are upstream!

They’re in different forests; there won’t be anywhere you can hg fclone from and get a buildable Zero until the two forests get promoted. But upstream is upstream!

Thank you Tom Rodriguez, Tim Bell, Andrew Hughes, and everyone else who made this happen :D

Syndicated 2009-10-15 13:56:19 from gbenson.net

Long overdue update

It’s been a while; time for a catchup!

June and July I mostly spent cleaning up Shark. HotSpot’s existing JITs, client and server, both inline pointers to objects in the native code they generate. These pointers need to be visible to the garbage collector, both so it knows the objects are live and so it can rewrite the pointers if it moves the object. This is trivial for client and server, as they both have access to the native code they generate: each method’s code is accompanied by a list of pointer locations within it. Shark, on the other hand, has no access to its generated native code other than knowing its address and size. Pointers can’t be inlined in Shark – it can’t tell the garbage collector where they are – so Shark had to load all garbage collected object pointers from other places, generally wherever the interpreter stored them.

This caused no end of problems. Aside from requiring more loads than the other JITs (at least one per object, and sometimes three or four) Shark had to mirror huge chunks of the interpreter. It had to cope with objects that were loaded in the VM (so the compiler could see them) but not cached in the interpreter (where the compiled code could see them). Because Shark wasn’t behaving like the other compilers, HotSpot’s compiler support layer would break in all kinds of exciting and imaginative ways. Finally, the method used by the server JIT to optimize interface calls to virtual calls and virtual calls to direct calls could not be used. Aside from the obvious speedup, Shark can only inline direct calls, so reducing virtual and interface calls to direct calls exposes them to the inliner. Calls in Shark have a lot of overhead, so this would have been a big win.

Sometime in May I figured out how to fake inlined object pointers. HotSpot’s compiler interface expects the compiler to generate native code into a CodeBuffer. Shark, of course, uses LLVM, which generates code into a buffer it allocates. Shark had a HotSpot code buffer, but it didn’t do a lot with it. Now, every time Shark has an object pointer to inline, it writes it into the HotSpot code buffer where the garbage collector can see it. The generated code then loads the object pointer from the code buffer whenever it needs it. The pointer is still not inlined – there’s still a load required – but now it’s always only one load. Not a big speedup in itself, but it meant the remaining interpreterisms could be removed, which fixed the support layer breakages and allowed me to copy the interface-virtual-direct call optimization code more or less directly from the server compiler. Everything got a lot more stable, a lot more clean, and a little bit faster in the bargain.

During August I began the (long!) process of preparing Zero for submission to OpenJDK proper. It took some time to get started, but the patch has now gone through a couple of cycles of being reviewed by the HotSpot team: the code has been reformatted, the build system has been almost completely rewritten, and a bunch of other things got changed. It’s still ongoing, but the HotSpot part of the patch seems close to acceptance and the much smaller remainder will hopefully be reviewed soon. I’ve been ramping up my testing with each step: this one bootstrapped and built itself on 32-bit x86, x86_64 and 32-bit PowerPC, and has bootstrapped itself and is in the process of building itself on 64-bit PowerPC and 64-bit zSeries.

Also in August, Ed Nevill released his assembler interpreter for ARM. It replaces part of Zero with hand-crafted assembly language, making OpenJDK 2-8 times faster on that platform.

After the Zero patch is accepted, my next task will be getting Zero certified on 64-bit zSeries. I won’t have a lot of time for Shark until that’s done, but I have one last thing I want to do before I step aside for a couple of months. Xerxes Rånby posted some benchmarks of Zero, Shark, and the assembler interpreter on ARM; Shark is gratifyingly faster than everything on five of the tests, but considerably slower than the assembler interpreter on the other four. I’m not happy with that!

On the tests where it’s slower, Shark is showing very little improvement over Zero, which suggests that these benchmarks are not spending a lot of time interpreting bytecode (which Shark would have compiled and made faster). I suspect these benchmarks are spending a lot of time in JNI calls. Back in February, Ed Nevill posted some profilies he had made to figure out why some interpreter improvements he had made had had very little effect; those profiles seemed to imply that the VM was spending a lot of its time setting up JNI calls. Zero uses libffi for this, and we at Red Hat have long suspected that libffi is slow.

HotSpot’s JITs have the capability to “compile” JNI methods. This sounds odd, as JNI methods are already native code; what’s actually getting compiled is the interface between the JVM and the native JNI code. If Shark could compile JNI methods, whenever HotSpot found a hot JNI method it would be able to replace its generic, one-size-fits-all interface code (using libffi) with an LLVM-generated interface custom built specifically for that method. I’m going to spend a week or so making Shark able to compile these methods, before I descend into zSeries TCK hell…

Syndicated 2009-10-09 16:20:37 from gbenson.net

First Shark self-builds

Xerxes Rånby and I simultaneously decided to try building Shark with Shark today… and both worked!

Syndicated 2009-06-10 14:37:10 from gbenson.net

Instrumenting Zero and Shark

Every so often I find myself adding little bits of code to Zero or Shark, to figure out obscure bugs or to see whether working on some optimization or another is worthwhile. I did it again today, and thought I’d write a little tutorial.

The first versions of Shark implemented a lot of things the same way as the interpreter, ie slowly. Since February I’ve been slowly replacing these interpreter-isms with implementations that are more compiler-like, and today there’s only one left: invokeinterface. The reason I left it until last is that it’s the biggest and the ugliest: it’ll no doubt be a pig to do, and quite frankly I don’t really want to do it. To see if I could get away with not bothering with it, I decided to instrument Shark so I could run SPECjvm98 and have it print out the number of times Shark-compiled code executed an invokeinterface for every benchmark.

First, I needed somewhere to store the counter. I decided to put it in the individual thread’s JavaThread objects as they’re easy to get at from both C++ and Shark, and they’re thread-specific so you don’t have to worry about locking.

diff -r 4cc0bc87aef4 ports/hotspot/src/os_cpu/linux_zero/vm/thread_linux_zero.hpp
--- a/ports/hotspot/src/os_cpu/linux_zero/vm/thread_linux_zero.hpp	Fri May 29 12:46:07 2009 +0100
+++ b/ports/hotspot/src/os_cpu/linux_zero/vm/thread_linux_zero.hpp	Fri May 29 14:44:04 2009 +0100
@@ -32,6 +32,25 @@
     _top_zero_frame = NULL;
   }

+ private:
+  int _interface_call_count;
+
+ public:
+  int interface_call_count() const
+  {
+    return _interface_call_count;
+  }
+  void set_interface_call_count(int interface_call_count)
+  {
+    _interface_call_count = interface_call_count;
+  }
+
+ public:
+  static ByteSize interface_call_count_offset()
+  {
+    return byte_offset_of(JavaThread, _interface_call_count);
+  }
+
  public:
   ZeroStack *zero_stack()
   {

So we have the field itself, a getter and setter to access it from C++, and a static method to expose the offset of the field in the thread object to Shark. Next we need to make Shark update the counter:

diff -r 4cc0bc87aef4 ports/hotspot/src/share/vm/shark/sharkTopLevelBlock.cpp
--- a/ports/hotspot/src/share/vm/shark/sharkTopLevelBlock.cpp	Fri May 29 12:46:07 2009 +0100
+++ b/ports/hotspot/src/share/vm/shark/sharkTopLevelBlock.cpp	Fri May 29 14:44:04 2009 +0100
@@ -985,6 +985,16 @@
 // Interpreter-style interface call lookup
 Value* SharkTopLevelBlock::get_interface_callee(SharkValue *receiver)
 {
+  Value *count_addr = builder()->CreateAddressOfStructEntry(
+    thread(),
+    JavaThread::interface_call_count_offset(),
+    PointerType::getUnqual(SharkType::jint_type()));
+  builder()->CreateStore(
+    builder()->CreateAdd(
+      builder()->CreateLoad(count_addr),
+      LLVMValue::jint_constant(1)),
+    count_addr);
+
   SharkConstantPool constants(this);
   Value *cache = constants.cache_entry_at(iter()->get_method_index());

We’re almost ready to add the SPECjvm98-specific bits now, but there’s one thing left. Some of the benchmarks are multithreaded, but we have one counter per thread; we need a way to set and get the counters from all running threads. HotSpot has some code to iterate over all the threads in the VM, but it’s all private to the Threads class. Not to worry though, we’ll just stick it in there:

diff -r 4cc0bc87aef4 openjdk-ecj/hotspot/src/share/vm/runtime/thread.hpp
--- a/openjdk-ecj/hotspot/src/share/vm/runtime/thread.hpp	Fri May 29 12:46:07 2009 +0100
+++ b/openjdk-ecj/hotspot/src/share/vm/runtime/thread.hpp	Fri May 29 14:44:04 2009 +0100
@@ -1669,6 +1669,9 @@
   // Deoptimizes all frames tied to marked nmethods
   static void deoptimized_wrt_marked_nmethods();

+ public:
+  static void reset_interface_call_counts();
+  static int  interface_call_counts_total();
 };

diff -r 4cc0bc87aef4 openjdk-ecj/hotspot/src/share/vm/runtime/thread.cpp
--- a/openjdk-ecj/hotspot/src/share/vm/runtime/thread.cpp	Fri May 29 12:46:07 2009 +0100
+++ b/openjdk-ecj/hotspot/src/share/vm/runtime/thread.cpp	Fri May 29 14:44:04 2009 +0100
@@ -3828,6 +3828,21 @@
   }
 }

+void Threads::reset_interface_call_counts()
+{
+  ALL_JAVA_THREADS(thread) {
+    thread->set_interface_call_count(0);
+  }
+}
+
+int Threads::interface_call_counts_total()
+{
+  int total = 0;
+  ALL_JAVA_THREADS(thread) {
+    total += thread->interface_call_count();
+  }
+  return total;
+}

 // Lifecycle management for TSM ParkEvents.
 // ParkEvents are type-stable (TSM).

Now we’re ready to add some SPECjvm98-specific code. A quick poke around in SPECjvm98 brings up the method spec.harness.ProgramRunner::runOnce as a likely place to hook ourselves in. This will be run by the interpreter — Shark won’t compile it as it’s only called a few times — so we put our code into the C++ interpreter’s normal entry which is the bit that executes bytecode methods:

diff -r 4cc0bc87aef4 ports/hotspot/src/cpu/zero/vm/cppInterpreter_zero.cpp
--- a/ports/hotspot/src/cpu/zero/vm/cppInterpreter_zero.cpp	Fri May 29 12:46:07 2009 +0100
+++ b/ports/hotspot/src/cpu/zero/vm/cppInterpreter_zero.cpp	Fri May 29 14:44:04 2009 +0100
@@ -42,6 +42,26 @@
   JavaThread *thread = (JavaThread *) THREAD;
   ZeroStack *stack = thread->zero_stack();

+  char *benchmark = NULL;
+  {
+    ResourceMark rm;
+    const char *name = method->name_and_sig_as_C_string();
+    if (strstr(name, “spec.harness.ProgramRunner.runOnce(”) == name) {
+      intptr_t *locals = stack->sp() + method->size_of_parameters() - 1;
+      if (LOCALS_INT(5) == 100) {
+        Threads::reset_interface_call_counts();
+
+        name = LOCALS_OBJECT(1)->klass()->klass_part()->name()->as_C_string();
+        const char *limit = name + strlen(name);
+        while (*(–limit) != ‘/’);
+        const char *start = limit;
+        while (*(–start) != ‘/’);
+        start++;
+        benchmark = strndup(start, limit - start);
+      }
+    }
+  }
+
   // Adjust the caller’s stack frame to accomodate any additional
   // local variables we have contiguously with our parameters.
   int extra_locals = method->max_locals() - method->size_of_parameters();
@@ -59,6 +79,12 @@

   // Execute those bytecodes!
   main_loop(0, THREAD);
+
+  if (benchmark) {
+    tty->print_cr(”%s: %d interface calls”,
+                  benchmark, Threads::interface_call_counts_total());
+    free(benchmark);
+  }
 }

 void CppInterpreter::main_loop(int recurse, TRAPS)

This looks a little messy, but what it’s basically doing is spotting calls to spec.harness.ProgramRunner::runOnce and extracting the name of the benchmark from its arguments. It is complicated by the fact that SPECjvm98 intersperses full runs with hidden tenth-speed ones, so we use the speed argument (in LOCALS_INT(5)) to ignore the hidden runs.

Now we’re ready to run the benchmarks and see what happens:

SPECjvm98 results with instrumented VM

Looks like making invokeinterface faster is worthwhile after all! Now all I have to do is do it ;)

Syndicated 2009-05-29 15:15:04 from gbenson.net

Zero and Shark article

This past month or so I’ve been working on an article about Zero and Shark for java.net. It went live today, so if you fancy a little primer on what Zero and Shark are and how they work then head over there and check it out :)

Syndicated 2009-05-27 16:07:09 from gbenson.net

Debugging the C++ interpreter

Every so often I find myself wanting to add debug printing to the C++ interpreter for specific methods. I can never remember how I did it the last time and have to figure it out all over again, so here’s how:

diff -r 4d8381231af6 openjdk-ecj/hotspot/src/share/vm/interpreter/bytecodeInterpreter.cpp
--- a/openjdk-ecj/hotspot/src/share/vm/interpreter/bytecodeInterpreter.cpp  Tue Apr 21 09:50:43 2009 +0100
+++ b/openjdk-ecj/hotspot/src/share/vm/interpreter/bytecodeInterpreter.cpp  Wed Apr 22 11:13:35 2009 +0100
@@ -555,6 +555,15 @@
          topOfStack stack_base(),
          “Stack top out of range”);

+  bool interesting = false;
+  if (istate->msg() != initialize) {
+    ResourceMark rm;
+    if (!strcmp(istate->method()->name_and_sig_as_C_string(),
+                “spec.benchmarks._202_jess.jess.Rete.FindDeffunction(Ljava/lang/String;)Lspec/benchmarks/_202_jess/jess/Deffunction;”)) {
+      interesting = true;
+    }
+  }
+
   switch (istate->msg()) {
     case initialize: {
       if (initialized++) ShouldNotReachHere(); // Only one initialize call

The trick is getting the fully-qualified name of the method right: the method name contains dots, but the class names in its signature contain slashes. You’re there once you have that down.

Syndicated 2009-04-22 10:27:19 from gbenson.net

I’m not dead

I haven’t blogged for a while. I’ve been working on Shark’s performance, walking through the native code generated for critical methods and looking at what’s happening. There’s several cases where I can see that some piece of code is unnecessary, but translating that into a way that Shark can see it’s unnecessary is non-trivial. I’m thinking I may need to separate the code generation, adding an intermediate layer between the typeflow and the LLVM IR so I can add things which are maybe necessary and then remove them if not. It all seems a bit convoluted — bytecode → typeflow → new intermediate → LLVM IR → native — but the vast bulk of the Shark’s time is spent in the last step so a bit more overhead to create simpler LLVM IR should speed up compilation as well as the runtime.

None of this has been particularly bloggable, but I wanted to point out two exiting things that are happening in Shark land. Robert Schuster and Xerxes Rånby have been busy getting Shark to run on ARM, and Neale Ferguson has started porting LLVM to zSeries with the intention of getting Shark running there. I expected to see Shark on ARM sooner or later, but Shark on zSeries came completely out of the blue. I’m really looking forward to seeing that happen!

Syndicated 2009-04-08 10:31:46 from gbenson.net

Good news and bad news

Bad news first. The drop in speed between the Zeros in IcedTea6 1.3 and 1.4 doesn’t seem to come from Zero itself. I did a build of IcedTea6 1.4 with everything in ports/hotspot/src/*cpu reverted to 1.3, and the speed loss remained. It must be something to do with the newer HotSpot, or some other patch that got added or changed. I don’t really want to spend any more time on this than I have, so we’ll just have to live with it.

I’ve not come to any conclusions as to the difference in speed between the native-layer C++ interpreter and Zero either. It’s not the unaligned access stuff I mentioned: I ran some benchmarks, but the results were ambiguous. It may be libffi, but again, I don’t want to spend more time on this…

The good news is that I’ve been checking the Zero sources for SCA cover, emailing various people, and there’s only one tiny easily-removable bit I’m unsure about. I spent the morning preparing and submitting the first of the patches that will be required, the core build patch, which will hopefully be well received.

Syndicated 2009-02-20 14:53:12 from gbenson.net

18 Feb 2009 (updated 19 Feb 2009 at 15:30 UTC) »

Benchmarks

Advogato can't display this entry, read it here instead.

Syndicated 2009-02-18 17:19:29 from gbenson.net

240 older entries...

New Advogato Features

New HTML Parser: The long-awaited libxml2 based HTML parser code is live. It needs further work but already handles most markup better than the original parser.

Keep up with the latest Advogato features by reading the Advogato status blog.

If you're a C programmer with some spare time, take a look at the mod_virgule project page and help us with one of the tasks on the ToDo list!