Older blog entries for braden (starting at number 114)

Exception-safe management of JNI local references

…or, solving the problem of how to delete local references when the execution context isn’t clear.

JNI includes notions of “local” and “global” references. Loosely speaking, local references correspond to those that are local to the scope of a function; and global references correspond to those that persist outside the scope of a function. When Java calls a native method, it provides the native code with at stack frame where local references can be stored. The native code can then proceed to make calls to JNI functions which, in general, return local references. These local references are automatically stored in the stack frame; and the Java runtime takes care of cleaning them up when it pops the stack frame upon leaving the native method implementation.

So far, so good. But not all JNI function calls occur in response to Java calling a native method implementation. In fact, if you’re starting up the VM via JNI, you probably end up calling JNI functions that return local references just the same. Only in this case, the Java runtime hasn’t provided you with a stack frame that it will magically clean up once your code is done executing. Instead, you’ll need to delete the local references explicitly.

Thus we have two very different ways local references must be handled, depending on the context of the JNI function calls. And the inevitable problem: in a utility function which might be called from either context, how should intermediate local references be handled? Consider a function that creates a java.net.URL instance:

jobject create_url(JNIEnv & env, const char * const url)
{
    const jstring url_string = env.NewStringUTF(url);
    if (!url_string) {
        env.ExceptionClear();
        throw std::runtime_error("failed to construct string for URL");
    }

    const jclass url_class = env.FindClass("java/net/URL");
    if (!url_class) {
        env.ExceptionClear();
        throw std::runtime_error("could not find java.net.URL class");
    }

    const jmethodID ctor_id =
        env.GetMethodID(url_class, "", "(Ljava/lang/String;)V");
    if (!ctor_id) {
        env.ExceptionClear();
        throw std::runtime_error("failed to get "
                                 "java.net.URL.URL(java.lang.String) "
                                 "constructor");
    }

    const jobject url_obj =
        env.NewObject(url_class, ctor_id, url_string);
    if (!url_obj) {
        env.ExceptionClear();
        throw std::runtime_error("could not create java.net.URL "
                                 "instance");
    }

    return url_obj;
}

The above code will work just fine when it is called from a native method implementation. But if it is called outside that context, it will leak the local references corresponding to url_string and url_class. (We can assume the caller has responsibility for the local reference corresponding to url_obj in both cases.)

So, let’s toss in the code to delete the local references. We need to be exception-safe, so let’s use ScopeGuard:

jobject create_url(JNIEnv & env, const char * const url)
{
    const jstring url_string = env.NewStringUTF(url);
    if (!url_string) {
        env.ExceptionClear();
        throw std::runtime_error("failed to construct string for URL");
    }
    scope_guard url_string_guard =
        make_obj_guard(env, &JNIEnv::DeleteLocalRef, url_string);

    const jclass url_class = env.FindClass("java/net/URL");
    if (!url_class) {
        env.ExceptionClear();
        throw std::runtime_error("could not find java.net.URL class");
    }
    scope_guard url_class_guard =
        make_obj_guard(env, &JNIEnv::DeleteLocalRef, url_class);

    const jmethodID ctor_id =
        env.GetMethodID(url_class, "", "(Ljava/lang/String;)V");
    if (!ctor_id) {
        env.ExceptionClear();
        throw std::runtime_error("failed to get "
                                 "java.net.URL.URL(java.lang.String) "
                                 "constructor");
    }

    const jobject url_obj =
        env.NewObject(url_class, ctor_id, url_string);
    if (!url_obj) {
        env.ExceptionClear();
        throw std::runtime_error("could not create java.net.URL "
                                 "instance");
    }

    return url_obj;
}

There. Now we can call the function outside a native method implementation. But in making that work, we’ve made the function unusable from within a native method implementation. Clearly we need to make the calls to JNIEnv::DeleteLocalRef conditional; and we’ll have to let the caller tell us what context the function is being called in.

jobject create_url(JNIEnv & env,
                    const char * const url,
                    const bool delete_local_refs)
{
    const jstring url_string = env.NewStringUTF(url);
    if (!url_string) {
        env.ExceptionClear();
        throw std::runtime_error("failed to construct string for URL");
    }
    scope_guard url_string_guard =
        make_obj_guard(env, &JNIEnv::DeleteLocalRef, url_string);
    if (!delete_local_refs) { url_string_guard.dismiss(); }

    const jclass url_class = env.FindClass("java/net/URL");
    if (!url_class) {
        env.ExceptionClear();
        throw std::runtime_error("could not find java.net.URL class");
    }
    scope_guard url_class_guard =
        make_obj_guard(env, &JNIEnv::DeleteLocalRef, url_class);
    if (!delete_local_refs) { url_class_guard.dismiss(); }

    const jmethodID ctor_id =
        env.GetMethodID(url_class, "", "(Ljava/lang/String;)V");
    if (!ctor_id) {
        env.ExceptionClear();
        throw std::runtime_error("failed to get "
                                 "java.net.URL.URL(java.lang.String) "
                                 "constructor");
    }

    const jobject url_obj =
        env.NewObject(url_class, ctor_id, url_string);
    if (!url_obj) {
        env.ExceptionClear();
        throw std::runtime_error("could not create java.net.URL "
                                 "instance");
    }

    return url_obj;
}

Well, there we are. It’s exception-safe and the caller can tell it the Right Thing to do. But…It sure does Suck.

  • It relies on the caller telling it to do the right thing; which means it’s pretty easy to use wrong.
  • We wind up having to check delete_local_refs at each point where we might (or might not) need to call JNIEnv::DeleteLocalRefs. Our need for exception-safety prevents us from consolidating this logic near the end of the function. Consequently, the code is interspersed with yet more error handling logic that gets in the way of understanding the function’s primary logic when reading the code.

So, is there a better way? Fortunately, yes.

JNI provides functions that allow us to create (and destroy) stack frames for local references, JNIEnv::PushLocalFrame and JNIEnv::PopLocalFrame. So, instead of relying on a stack frame that may or may not be there depending on the calling context, we can just create our own, regardless.

jobject create_url(JNIEnv & env, const char * const url)
{
    if (env.PushLocalFrame(3) < 0) { return 0; }
    scope_guard local_frame_guard =
        make_obj_guard(env, &JNIEnv::PopLocalFrame, 0);

    const jstring url_string = env.NewStringUTF(url);
    if (!url_string) {
        env.ExceptionClear();
        throw std::runtime_error("failed to construct string for URL");
    }

    const jclass url_class = env.FindClass("java/net/URL");
    if (!url_class) {
        env.ExceptionClear();
        throw std::runtime_error("could not find java.net.URL class");
    }

    const jmethodID ctor_id =
        env.GetMethodID(url_class, "", "(Ljava/lang/String;)V");
    if (!ctor_id) {
        env.ExceptionClear();
        throw std::runtime_error("failed to get "
                                 "java.net.URL.URL(java.lang.String) "
                                 "constructor");
    }

    const jobject url_obj =
        env.NewObject(url_class, ctor_id, url_string);
    if (!url_obj) {
        env.ExceptionClear();
        throw std::runtime_error("could not create java.net.URL "
                                 "instance");
    }

    return url_obj;
}

Once again, we use ScopeGuard; only now we’re using it to call JNIEnv::PopLocalFrame. Note that we’ve provided space for three local references in our frame, corresponding to url_string, url_class, and url_obj.

But the above code is horribly broken: when we return from the function, the local reference associated with url_obj gets cleaned up with all the others! So, how do we get this reference out of our local frame and return it, unbound, to the caller?

The solution is a temporary global reference. We can convert url_obj to a global reference for the remaining duration of the local frame; and then convert it back to a local reference before returning to the caller. Once more, ScopeGuard is our friend:

jobject create_url(JNIEnv & env, const char * const url)
{
    using boost::ref;

    //
    // We can safely run DeleteGlobalRef in the scope guard because
    // calling DeleteGlobalRef with 0 is a no-op.
    //
    jobject result_global_ref = 0;
    scope_guard result_global_ref_guard =
        make_obj_guard(env,
                       &JNIEnv::DeleteGlobalRef,
                       ref(result_global_ref));
    {
        if (env.PushLocalFrame(3) < 0) { return 0; }
        scope_guard local_frame_guard =
            make_obj_guard(env, &JNIEnv::PopLocalFrame, 0);

        const jstring url_string = env.NewStringUTF(url);
        if (!url_string) {
            env.ExceptionClear();
            throw std::runtime_error("failed to construct string for URL");
        }

        const jclass url_class = env.FindClass("java/net/URL");
        if (!url_class) {
            env.ExceptionClear();
            throw std::runtime_error("could not find java.net.URL class");
        }

        const jmethodID ctor_id =
            env.GetMethodID(url_class, "", "(Ljava/lang/String;)V");
        if (!ctor_id) {
            env.ExceptionClear();
            throw std::runtime_error("failed to get "
                                     "java.net.URL.URL(java.lang.String) "
                                     "constructor");
        }

        const jobject url_obj =
            env.NewObject(url_class, ctor_id, url_string);
        if (!url_obj) {
            env.ExceptionClear();
            throw std::runtime_error("could not create java.net.URL "
                                     "instance");
        }

        //
        // Create a global reference so that the new object will outlive
        // the local frame.
        //
        result_global_ref = env.NewGlobalRef(url_obj);
        if (!result_global_ref) { return 0; }
    }

    //
    // NewLocalRef does not throw any Java exceptions.
    //
    const jobject result = env.NewLocalRef(result_global_ref);
    if (!result) { throw std::bad_alloc(); }

    return result;
}

Ta da. We introduce an additional block scope for the duration of our local stack frame. We propagate the reference to the URL class instance out of the frame’s lifetime by creating an additional global reference. Once the frame is destroyed, this global reference is all that’s left. We then create a new local reference to return to the caller, and let a scope guard take care of cleaning up the global reference.

Syndicated 2008-01-21 05:08:59 from endoframe :: log

We’ve got Spirit

I have spent a large chunk of my free time over the last several months replacing OpenVRML’s ANTLR parsers with ones using Spirit. This was my first attempt to do something nontrivial with Spirit. So this comment from the author of Spirit absolutely made my day.

Syndicated 2007-12-15 05:27:24 from endoframe :: log

1 Dec 2007 (updated 2 Jan 2008 at 09:08 UTC) »

New Autoconf OpenGL macros

After nearly a year since pushing this project to Google’s project hosting, I’ve finally made a release tarball of my Autoconf macros for OpenGL. This was motivated mostly by significant changes that were precipitated by requirements for building on Mac OS X 10.5.

10.5 is a bit of a mixed bag relative to its predecessor as far as building with OpenGL is concerned. The good: they finally fixed the longstanding bug with the GLU tesselator callback function type being interpreted incorrectly. The bad: linking with X11 now requires some extremely goofy linker flags.

One nice thing, though, is that I was finally able to get rid of the annoying ‑‑with‑apple‑opengl‑framework option.

Syndicated 2007-11-30 23:13:57 (Updated 2007-11-30 23:16:41) from endoframe :: log

1997 called…

I have begun to wonder just what it would take to create an ActiveX control for OpenVRML. It has been about a decade since I touched COM. At the time, I had just (barely) learned C++. And I was struck then that just about every Good Practice for C++ I was learning about was being flagrantly [...]

Syndicated 2007-11-12 03:41:30 from endoframe :: log

Nokia N75

I’ve used Motorola mobile phones since I’ve had a mobile phone, starting with the V60i back in 2002 (as a customer of the original AT&T Wireless), followed with a V551 (with Cingular), which was in turn replaced with a RAZR V3 (upon its untimely demise). The latter two phones had nearly identical software; but I was [...]

Syndicated 2007-09-10 03:04:02 from endoframe :: log

Consolation prize

I discovered Consolas a couple of weeks ago; and I’ve been using it heavily since then. It renders nicely on the Mac (where it has supplanted Andale Mono) as well as with Xft (where it has supplanted DejaVu Sans Mono). Something about Consolas makes Emacs behave just a little bit goofy: for some reason windows sometimes [...]

Syndicated 2007-08-06 20:12:02 from endoframe :: log

A snake in a blender

I didn’t want to like Python. I’m still not sure that I do. But I have to admit that it is starting to grow on me. I’m beginning to get used to its goofy ideas about whitespace. And its double-underscore-emblazoned special identifiers are starting to hurt my eyes less. I have been learning to use Blender. [...]

Syndicated 2007-07-22 07:41:14 from endoframe :: log

18 Jun 2007 (updated 2 Jan 2008 at 09:08 UTC) »

Shedding ANTLR

My word… was it really all the way back in 2001 that I first latched onto the idea of replacing OpenVRML’s ANTLR-based parser with one written using Spirit?

I guess I shouldn’t be so surprised; I was immediately taken with Spirit upon discovering it. Though back in 2001 Spirit still had a good deal of growing up to do; it’s come a long way since then. And given that Joel de Guzman and his cohorts are hard at work on Spirit 2, I get the idea that it will yet go a good deal farther. But I digress.

It’s the middle of 2007 and I’ve gotten serious about writing a VRML parser for OpenVRML using Spirit. Why now? Well, ANTLR certainly hasn’t gotten to be any less of an annoying dependency over the years. But another factor is that a new major version of ANTLR has been released (3.0). I think my efforts are better spent moving away from ANTLR entirely than on upgrading to the new version (which I understand to include nontrivial changes to the grammar format).

Now, I don’t want to come across as disliking ANTLR. It’s a really nice tool. In fact, it’s the nicest parser generator I’ve ever used. Functionality is very discoverable and I found, for the most part, the general behavior of ANTLR parsers to be very intuitive. But it has its downsides:

  • It’s a code generator, and thus it has the caveats that go with any code generation.
  • It’s a Java program, and thus it has the caveats that go with any Java program.
  • Even though it can generate code for a number of languages that are Not Java, the Not Java language backends are maintained by persons other than the primary author of ANTLR; thus, these languages wind up being second class citizens. (For example, ANTLR 3.0 has been released without C++ support since the maintainer of the C++ backend for ANTLR 2.x didn’t have time to port it to the new version.)

I actually started in earnest on this project at the end of last summer. I made a good deal of headway, getting as far as developing a good understanding of how to use Spirit’s stored_rule to create a grammar with productions that get modified as part of the parse. This solved the somewhat tricky issue of parsing node field values. But then I got side-tracked with getting the stand-alone viewer (openvrml-player) working reliably; that took much, much longer than I’d anticipated. But now I’ve picked up pretty much where I left off. As of this writing, I can parse nodes, EXTERNPROTOs, and PROTOs, except for IS mappings. I still have to do ROUTEs; though they will be pretty easy now that I’ve got DEF working.

As with the ANTLR-based parser, I’m doing a good deal of semantic checking; this parser will be just as aggressive about rejecting code that’s Not Quite Right as OpenVRML’s current parser. But unlike OpenVRML’s current parser, I’m using very little of OpenVRML’s runtime machinery to accomplish this checking. The idea is to make this parser much more reusable than OpenVRML’s current parser. The current parser isn’t really exposed; users can read the file into the runtime and then inspect the node tree created for the runtime. It turns out, though, that a good deal of OpenVRML’s users (and prospective users) don’t care one bit about a VRML/X3D runtime—they just want to read a VRML or X3D file and do something with the data. So, the new parser will have

  • Pluggable semantic actions
  • Minimal dependencies on the rest of libopenvrml—ideally, linking with libopenvrml won’t be required at all for someone just using the parser

This will all be possible through the magic of Spirit and C++ templates.

Syndicated 2007-06-18 05:21:22 (Updated 2007-07-02 18:43:10) from endoframe :: log

30 May 2007 (updated 2 Jan 2008 at 09:08 UTC) »

iRiver S10

After years of sitting on the sidelines, I’ve taken the plunge and gotten a digital audio player.

iPods don’t interest me. While I appreciate their slick interface, I’m not the least bit interested in iTunes (or other online music services). And the fact that they don’t play Ogg is a deal-breaker for me. I use a Linux desktop (GNOME) and Ogg is easy for me to use, high quality, and completely unencumbered.

The requirement for Ogg narrows the field quite a bit. Narrowing it further, I’m interested in a flash-based device. So for some weeks I’ve been considering the Cowon D2 or the iRiver Clix. But ultimately the iRiver S10 won me over. The multimedia capabilities of the D2 and Clix don’t interest me; and the S10’s form factor is quite attractive.

I received my S10 yesterday. The functionality and quality are top-notch. And Nautilus makes it easy for me to convert selections from my library of FLACs to Oggs on the S10.

I am very happy with it. But here are the few things I don’t like:

  • Unless you’re using a playlist, files play in the order of their FAT entries. I’d have preferred/expected alphanumeric order. The included software includes the facility to reorder the entries (and thus play order) however you like; however…
  • The included software only works on Windows. I realize that providing software that works on Linux is probably expecting too much; but I was expecting a Mac version. This software isn’t absolutely vital; but the FAT entry reordering is very useful and the software is the most convenient means I’ve found of creating playlists. (Though so far I haven’t found myself messing with playlists.)
  • The included software seems not to work at all unless it’s run by a user with Administrator privileges. Lame. Really lame.
  • No case was included; nor does anyone appear to make one. Hopefully that will change. An armband would be especially handy.
  • The “D-click” navigation is a pleasure to use; but it means that the player can’t really be operated while stored somewhere that’s at all a tight fit—like, for instance, pockets in jeans. If there is some way to lock the interface, I haven’t discovered it yet.

Syndicated 2007-05-30 21:59:51 (Updated 2007-06-18 06:28:47) from endoframe :: log

Feeding (and silencing) the beast

My primary development box, hinge, has seen a bit of an overhaul lately. Most significantly, I snagged some Opteron 285s that were refurbished units (i.e., system pulls) from HP servers for $325 each. These replaced the 242s that I originally built hinge with. Twice the cores and a full 1 GHz faster—quite an improvement. hinge is housed [...]

Syndicated 2007-05-02 06:36:24 from endoframe :: log

105 older entries...

New Advogato Features

New HTML Parser: The long-awaited libxml2 based HTML parser code is live. It needs further work but already handles most markup better than the original parser.

Keep up with the latest Advogato features by reading the Advogato status blog.

If you're a C programmer with some spare time, take a look at the mod_virgule project page and help us with one of the tasks on the ToDo list!