Log in

Accessing Native Maths Libraries

27 June 2012 By

What was said last

My last post was about how we want to test our maths libraries by accessing native library code with fuzzy data sets and then comparing results. The post before that was about how to access native code with minimum overhead. This post is about the actual implementation of our wrapper to native code that allows us access to native maths libraries.

What to wrap?

The prime candidates for wrapping are BLAS, LAPACK and SLATEC API calls; these libraries cover the majority of the core numerical methods, particularly in the field of linear algebra. All the libraries are written in Fortran 77 and the NETLIB reference libraries for BLAS and LAPACK are widely available and usually come in the form of ATLAS on Linux machines. Needless to say, I develop code solely on Linux variants and this is reflected in the following; however, there is no reason why these techniques couldn't be used on other OS's. First thing to note, we are aware of JBLAS, which wraps BLAS and LAPACK from ATLAS. However we want something that supports multithreading (especially with regards to exception handling), uses faster access methods, and importantly is a flexible tool for wrapping other Fortran libraries.

How we do it

To begin with, we note that the source for the reference BLAS, LAPACK and SLATEC libraries from Netlib are organised with one function per file. So first we need to extract the type information and routine names from the files. As fun as writing an F77 parser would be, an easier method is to use GCC to parse the code and dump out the SSA tree from which a bit of regex is all that is needed to extract the required information. This method also has the advantage of simply failing when called on Fortran code that isn't actually valid! For example purposes we'll use the "dgemv.f" code from BLAS. First run gcc:

#>gcc -c -fdump-tree-ssa dgemv.f

which gives a file called "dgemv.f.017t.ssa", the top of which contains:

;; Function dgemv (dgemv_)

dgemv (character(kind=1)[1:1] & restrict trans, integer(kind=4) & restrict m, integer(kind=4) & restrict n, real(kind=8) & restrict alpha, real(kind=8)[0:] * restrict a, integer(kind=4) & restrict lda, real(kind=8)[0:] * restrict x, integer(kind=4) & restrict incx, real(kind=8) & restrict beta, real(kind=8)[0:] * restrict y, integer(kind=4) & restrict incy, integer(kind=4) _trans) {

which is exactly what is needed: all the type information is there in an easily parsable form.

To make a convenient call to native DGEMV from Java, we need to have:

  1. Some C/C++ prototypes for the Fortran library calls
  2. Some C/C++ JNICALL code that moves/formats data and makes calls to the native Fortran libraries
  3. Some Java code that provides access to the C/C++ code once wrapped in a library
  4. Some Java signatures that describe the JNICALLs to Java

So we wrote some Python code to call GCC and extract the type information. The Python code contains maps between the various types in Fortran, C and Java, which can then be looked up when generating the items on the list above.

Some C/C++ prototypes for the Fortran library calls

The code we generated to tell C the prototype of the Fortran DGEMV function is given below.

In Fortran all arguments to functions are passed by reference, hence everything being a pointer, we need to account for this further up stream in the call chain. We also define DGEMV_F77 using the F77_FUNC macro to deal with variations in capitalisation and underscoring in function names used by different compilers.

Some C/C++ JNICALL code that marshalls data and makes calls to the native Fortran libraries

We first define a few of macros to handle problems with getting data from the JVM. MACRO_CRITICALGETPOINTER() attempts to obtain access to the memory via (*env)->GetPrimitiveArrayCritical(). Should the memory access fail, the MACRO_FAILEDPOINTERGRAB() throws a Java Exception to indicate a failure has occurred; should this fail, an exit code is given. Finally the MACRO_CRITICALFREEPOINTER() frees the pinning on the memory or deallocates if the memory was copied.

The JNICALL is automatically generated based on the type information and the type maps defined in the Python code. Within the call we make sure arrays are passed with an additional parameter prefixed with _offset_ to emulate pointer behaviour from the Java side. This allows us to arbitrarily stride to a position in memory, much as would be done with an array slice in Fortran or a pointer offset in C.

Some Java code that provides access to the C/C++ code once wrapped in a library

For ease, we provide two methods for each call to a JNICALL function, one with the array offsets and one without. Below is the call with offsets available. We have some cpp defines available to support locking and thread safety if needed. Those familiar with compiler preprocessors will note that Java doesn't have one! We are using GCC's CPP and some GNU SED in a GNU Make rule to pretend. We call the functions in Java by their true names and then pass the data through to the "wrapped" function of the same name that is available via JNI.

Some Java signatures that describe the JNICALLs to Java

Finally, and possibly the easiest bit, is just defining a signature for the "wrapped" call provided by the native library.

If only that were all...

This phrase is here for people looking how to "Call Java from native code when the environment pointer is not available" and coming in from search engines.

What we have created above is a method with which to call the native libraries. What we need now is something to load said libraries, and this is provided by the System.loadLibrary() function. We also need to address a rather important problem too. The Fortran BLAS and LAPACK routines have their own error-handling mechanism provided by the function XERBLA. This function contains the Fortran key word "STOP", which does what it says: it stops execution and exits. If a bad call is made to a native maths library function which results in the thread running XERBLA, the thread exits and takes the JVM with it. So we need to provide an override, our own version of XERBLA, which when called handles the error and does not exit. Thankfully, all calls to XERBLA in the native libraries are followed by the "RETURN" key word, which causes the thread to immediately exit the function and return to the caller. Therefore if we manage to override XERBLA we can continue to run in the JVM.

To override a function when creating a wrapping library is relatively simple, one simply forces the link order of the library to link a library containing the override function before the library that is being wrapped. Then, when the wrapping library is dynamically loaded and the symbol is looked up, the overriding library symbols take preference. This all seems fine, however, when XERBLA is called the thread making the call is embedded deep into the land of the native maths library and so has no knowledge of the JVM, which is rather problematic were one to want to throw an exception. The way around this is to globally cache the JVM pointer in the library containing the XERBLA override function via the JNI_OnLoad()function mechanism. This is sketched out in the following code for an overriding library providing the XERBLA function:

We can see in this code that we have a global JavaVM pointer cache *JVMcache. The JNI_OnLoad mechanism, which is called when the native library is loaded, is then used to set the global JavaVM pointer cache. Then, should XERBLA be called from within BLAS/LAPACK, because of the link order our XERBLA override is called. Within our XERBLA we then use the global JavaVM pointer cache and attach our running thread back to the JVM to get access to the environment pointer from which it is then trivial to "gracefully" handle the exception.

Anything else?

We automated all this code generation via a Python script. The whole lot was then put into an Autotools package. We wrote some new rules and macros to allow building and preprocessing Java code, which are rather useful for obvious reasons. The Java code was then zipped in a jar and the libraries are provided at system level accessed via the classpath variable.

Conclusion

We now have access via a relatively safe mechanism to native maths libraries. This allows us to test our code, and the code of others against reference implementations from Java directly. This means writing code to automate fuzzy data testing is possible as comparisons can easily be made, making our maths libraries more robust and heavily tested.

This is the developer blog of OpenGamma. For more posts by the OpenGamma team, check out our main blog.

About the Author

Stuart Archibald

Stuart Archibald is an Applied Mathematics Developer at OpenGamma.

Follow us on Twitter