JNI, Maven and OSGi

I've done my best to avoid JNI like the plague until now. Usually I would pick one side of the fence and code in either pure C or pure Java. This is my first attempt to do both, and after quite a steep learning curve it doesn't seem any more attractive. However it could certainly be worse. Thanks to a fantastic and seemingly magical build tool, Maven, and a couple of well configured plugins, I was able to create an OSGi compliant bundle including a native library wrapper for the Linux Robotics Framework.

The goal of this whole excercise is to produce a single jar file which contains a native library, can be deployed in an OSGi framework, and ofcourse work when other bundles try to access methods with native hooks.

Those of you that have dabbled with JNI, and everyone else who writes any native code will know that to load a native library requires the binaries to be visible to the OS and not hidden away inside a jar. I was very happy to discover that an OSGi framework is nice enough to unpack native libraries to the file system for you, and then point the JVM to them when your bundle tries to load them.

OSGi bundles are usually expected to be portable and able to be deployed within any framework instance. Adding native libraries to a bundle does tie them to a particular OS and processor architecture. The OSGi specification does however, include a Bundle-NativeCode header which allows developers to notify the OSGi framework of native libraries included in the bundle, as well as the environments they require to run. Multiple native binaries can be included in a bundle, each with their own dependencies and the framework will pick and unpack to correct one. So we can recover some of our bundle's portability by including native libraries for different environments. If there is no binary provided for the environment in question, the bundle will gracefully refuse to load with the message:

org.osgi.framework.BundleException: Unable to select a native library clause.

This about as close as you can get to complete JVM portability when using any JNI.

To achieve this feat I'm using Maven with two explicit plugins. The freehep-nar-plugin will process your java source code with javah to produce some header files. Then it will compile you implementation of these headers and package them into a shared library. It will also create a pile of meta-data which comes in handy when you want to distribute things. Unfortunately, this plugin packages your java interface (jar) separately from the native libraries (nar). While this is a good thing and allows much more flexibility when managing these artifacts, it's not what we want when deploying to an OSGi framework.

The maven-bundle-plugin is very helpful when creating OSGi bundles. Although inserting manifest headers is not that difficult, it can be quite tedious and incredibly difficult to debug if you get it wrong. This plugin will process your compiled classes and calculate all the required import package headers, based on their dependencies. It will also verify your bundles manifest to ensure all is well.

Both of these plugins expect to create an artifact of their own type. To get the result needed, we need to get a bit creative. The final packaging we want is a bundle, so this is what we specify in our projects pom.xml. This means that the nar-plugin goals need to be mentioned explicitly. Add this between your nar-plugin tags.

<executions>
<execution>
  <goals>
    <goal>nar-javah</goal>
    <goal>nar-compile</goal>
    <goal>nar-system-generate</goal>
    <goal>nar-integration-test</goal>
  </goals>
</execution>
</executions>

That will generate the headers, compile the source code, create a convenient little class to load the library and run the integration test. It will not however, create the nar artifact, which is fine by me.

This will now give us a shared library built in our target directory, so now we need to include it in our bundle. Insert this between your build tags.

<resources>
<resource>
  <directory>target/nar</directory>
  <includes><include>lib/**</include></includes>
  <excludes><exclude>*.flag</exclude></excludes>
</resource>
</resources>

This will include all native libraries as resources in the final jar.

Finally we need to add meta-data to the jar so an OSGi framework knows what to do with it. Add this between your plugins tags.

<plugin>
<groupId>org.apache.felix</groupId>
<artifactId>maven-bundle-plugin</artifactId>
<extensions>true</extensions>
<configuration>
  <instructions>
    <Export-Package>com.osgibots.lrf4j</Export-Package>
    <Private-Package></Private-Package>
    <Bundle-Activator>com.osgibots.lrf4j.Activator</Bundle-Activator>
    <Bundle-NativeCode>
      lib/i386-Linux-g++/jni/liblrf4j-0.1-SNAPSHOT.so;
        osname=Linux;processor=x86
    </Bundle-NativeCode>
  </instructions>
</configuration>
</plugin>

Take note of the Bundle-NativeCode header. This specifies the native libraries available inside the bundle and which environment they are intended to run on. A list of canonical names is available within the OSGi specification.

The OSGi specification mentions the fact that "...only one class loader can load a native library as specified by an absolute path. Loading of a native library file by multiple class loaders (from multiple bundles, for example) will result in a linkage error." This is important to note as most implementations of an JNI involve static initializers to call System.loadLibrary(). If you've read my last post on static variables in OSGi, you'll know this will cause issues.

The easiest thing to do is to load the native library from your bundles activator.

public class Activator implements BundleActivator {

  public void start(BundleContext arg0) throws Exception {
      NarSystem.loadLibrary();
  }

  public void stop(BundleContext arg0) throws Exception {

  }
}

Note the usage of NarSytem.loadLibrary(). This helps us avoid changing constants within the source code with every new minor release.

I've skipped an awful lot of detail on the usage of these two plugins. You can get you hands on my example via my Mercurial repository:

hg clone http://hg.osgibots.com/lrf4j

This is an JNI wrapper for the Linux Robotics Framework which you will need available on your LD_LIBRARY_PATH if you hope to build the example. Of course you can just browse the code online.

There are other approaches to combining these two plugins. The ideas above work best when you are only planning to deploy to one environment, and therefore, only need to compile the native library once. If you want to include binaries for other environments, you will be better off creating two projects. One project to create the nar artifacts for various environments and deploy them to a repository. The second is to fetch all those artifacts and compile them into a single bundle. Unless you can cross compile all your target binaries on one machine, you'll have a great time keeping your code base in sync...