JNI wrapper compilation
I now have a version of Shark with a basic implementation JNI wrapper compilation. Sadly I can’t say if it’s faster or not yet because it’s totally unstable!
The problem is this. When HotSpot wishes to compile a normal (interpreted) method, the thread initiating the compile simply adds it to a queue and carries on doing whatever it was it was doing. A separate thread, the compiler thread, loops over this queue, compiling methods one at a time. This means there’s only ever the one thread making LLVM calls, and everything is rosy.
When HotSpot wishes to compile a native (JNI) method, the thread initiating the compile bypasses the queue and does it immediately. It acquires a lock, the adapter hander library lock, so there’s only every one native method being compiled at once, but in the meantime the compiler thread is in all likelihood busy compiling some normal method or another, so there’s two threads making LLVM calls. LLVM doesn’t like this, not LLVM prior to 2.6 at any rate, and even then not without being written with a separate LLVMContext for each thread.
The obvious fix for this is for Shark to acquire a lock before compiling either a normal or a native method, ensuring that only one thread is calling into LLVM at once. This doesn’t work, however, as the compiler thread runs _thread_in_native. The benefit of this is that the compiler thread does not have to halt for safepoints (and the rest of the VM doesn’t have to wait for the compiler thread to halt) but the drawback of this is that threads running _thread_in_native may not own locks. You can’t make the compiler thread run other than _thread_in_native, not without losing the large chunk of the server compiler that Shark shares, and you can’t hack a lock in there anyway (by using a pthread mutex, say, rather than a HotSpot lock) because it’ll deadlock the first time a safepoint occurs when the compiler thread holds the lock (compiling a normal method) and a Java thread is blocking trying to take it (to compile a native one).
I’ve been circling around this issue for a couple of days now, but it looks like the only solution is to require LLVM 2.6 and rearrange everything to use separate contexts.