Scala and Rust interoperability via JNI

This is a post that I’ve been meaning to write already for some time. It’s based on work I did and a presentation I gave at our Scala guild meeting a few months ago. It explains how we can use Rust code from Scala (on JVM) via JNI with the help of sbt-jni. If you want to dive straight into example code, here’s the self contained example project.

Scala and Rust synergy

In our team, we use both Scala and Rust. Both are very powerful languages in their own regard, each with unique strengths (and also weaknesses). Whereas Scala is very high level, with a powerful type system and sits on the battle-tested JVM, Rust allows you to control low level details, enabling great CPU and/or memory efficiency without sacrificing safety. Thus, these two languages complement each other very well. We already have applications written in Scala, and some written in Rust, depending on which technology was better suited for the job. But can we have one application written in both languages?

Using native code from JVM

On the Java Virtual Machine – since we run Scala on JVM – we have a couple of options of how to call native code.

• JNI (Java Native Interface) is the official way. It is fast, but a bit clumsy, it requires programmers to write a lot of boilerplate interoperability code.
• JNA (Java Native Access) is relatively easier on programmers, but is slower, because it relies on dynamic behavior.
• The OpenJDK project Panama attempts to make it significantly easier to interoperate with native code, but is still in the experimental stage. (Check out SLinC to use it from Scala: GitHub repository and an introductory blogpost.)
• Of honorable mention is Scala Native, which compiles Scala to native code (in the same manner as is typical for Go or Haskell or Rust for that matter), allowing for direct use of native libraries. But it is also in the experimental phase of its life and, more importantly, by its nature the code doesn’t run on JVM, so it’s not relevant for our use case.

Because JNI is the official and most widely used option, with the benefits like most resources available online, easy googlability of issues, etc, I decided to explore it further. But the vast majority of articles or tutorials about JNI focus on calling C code from Java. There are also few resources and tools available for calling Rust from Java, or C from Scala. But it seemed like calling Rust from Scala hadn’t been done before. I wanted to change that – this very post is part of the effort.

Scala, as such, has support for using JNI, using the @native annotation, but it requires a lot of boilerplate code. Thankfully, there is sbt-jni, a suite of sbt plugins, libraries and macros, originally developed by Jakob Odersky, that make using JNI from Scala easier (helps with boilerplate on Scala side, boilerplate on the native side is still there). But it had only support for working with C and the CMake build tool. Adding support for Rust and Cargo (the build tool for Rust) was surprisingly easy! Unfortunately, at that time sbt-jni wasn’t maintained, but after some time, the project was taken over by Grigory Pomadchin and migrated under the github.com/sbt organization and the PR was merged afterall.

There is also a Rust library jni which makes it easier to work with JNI on the Rust side. We will use that too.

So let’s see how sbt-jni and jni help us to use Rust and Cargo with Scala.

Calling Rust code from Scala with the help of sbt-jni and the crate jni

To read all the details about how to use sbt-jni, have a look at its README. Here I’ll point out only the important bits relevant for Rust.

The sbt part

When working with sbt-jni, it is a good practice to have two sbt sub-projects. One is dedicated only to native code, Rust in our case. When packaged into a JAR, it will contain only the compiled .so library (on Linux). This is the sub-project that needs to have the plugin JniNative enabled. We should set the key sourceDirectory for task nativeCompile (this task comes from JniNative) to baseDirectory (in our case has value "scala-rust-interop/native"), because we want to have directory structure like this:

scala-rust-interop (the root)
├── native/
│   ├── Cargo.toml
│   ├── src/
│   │   ├── lib.rs

The sbt definition then should look like this:

// build.sbt
lazy val native = project
  .in(file("native"))
  .settings(
    nativeCompile / sourceDirectory := baseDirectory.value,
  )
  .enablePlugins(JniNative)Code language: Scala (scala)

The other sub-project is dedicated to the Scala wrapper around the native code. sbtJniCoreScope := Compile is an sbt-jni specific way to make sure that the core library will depend on the sbt-jni-core library which contains the class NativeLoader. More on NativeLoader later. When using sbt-jni, it is also a good practice to not depend on the native library, users of core should provide this dependency explicitly.

// build.sbt
lazy val core = project
  .in(file("core"))
  .settings(
    sbtJniCoreScope := Compile, // because we use `NativeLoader`, not the `@nativeLoader` macro
  )
  .dependsOn(native % Runtime)Code language: Scala (scala)

We don’t need to add any further Rust specific configuration to sbt, sbt-jni is clever enough to autodetect that we’re building a Rust project with Cargo if there is a Cargo.toml file in the directory nativeCompile / sourceDirectory points to. We’ll get to the content of the native Rust sub-project later, let’s first have a look at the Scala wrapper, the core sub-project.

The Scala part

We create a class that extends NativeLoader. This makes sure that the correct native library will be linked to the JVM process. The argument to NativeLoader should be the name of the library we want to wrap around, but without the lib prefix. Our class contains methods which don’t have definition, instead they are annotated with @native. When they’re called, the implementation in the respective native library will be executed. Note that the annotation doesn’t come from sbt-jni, but from Scala standard library.

sbt-jni also provides an alternative to extending NativeLoader, the @nativeLoader macro, but that doesn’t work on Scala 3, so we ignore it in this post. The NativeLoader way works on both Scala 2 and Scala 3.

// core/src/main/scala/com/github/sideeffffect/scalarustinterop/Divider.scala
package com.github.sideeffffect.scalarustinterop

import com.github.sbt.jni.syntax.NativeLoader

class Divider(val numerator: Int) extends NativeLoader("divider") {
  @native def divideBy(denominator: Int): Int // implemented in libdivider.so
}Code language: Scala (scala)

The Rust part

Let’s now get to the native sub-project. The content of Cargo.toml is pretty simple, we add the jni library and the resulting artifact is to be a dynamically linkable library (cdylib).

# native/Cargo.toml
[package]
name = "divider"
version = "0.1.0"
authors = ["John Doe <john.doe@gmail.com>"]
edition = "2018"

[dependencies]
jni = "0.19"

[lib]
crate_type = ["cdylib"]Code language: TOML, also INI (ini)

The content of our Rust library is the most elaborate part. Of utmost importance is the signature of the function(s). #[no_mangle] pub extern "system" makes a function callable as if it were written in C. The name itself and the parameters and return type have to follow the package and name of the class and the parameters of the respective method in question.

It may be tricky to write this manually from scratch, so we can use the help of the sbt-jni task javah. It generates C header files, which is a necessity when using JNI for C code. We write Rust, not C, but it’s easy to infer the expected Rust function signatures from the C prototypes.

> sbt javah
...
[info] Headers will be generated to .../scala-rust-interop/core/target/native/includeCode language: Bash (bash)

// core/target/native/include/com_github_sideeffffect_scalarustinterop_Divider.h
JNIEXPORT jint JNICALL Java_com_github_sideeffffect_scalarustinterop_Divider_divideBy
  (JNIEnv *, jobject, jint);Code language: C++ (cpp)

That prototype we translate to Rust like this:

#[no_mangle]
pub extern "system" fn Java_com_github_sideeffffect_scalarustinterop_Divider_divideBy(
  env: JNIEnv,
  object: JObject,
  denominator: jint,
) -> jintCode language: Rust (rust)

The first parameter JNIEnv is very important. Many JNI operations are done via it. Note that it is a wrapper from the Rust jni library, not directly a functionality from bare JNI itself. For example, accessing a field of a class is done like this:

env.get_field(object, "numerator", "I").unwrap().i().unwrap()Code language: Rust (rust)

object is the second parameter of our function, the object whose field we want to access. "numerator" is the name of the field and "I" is the type of the field ("I" stands for primitive int in Java bytecode). Note how both are strings and so are not statically verified by the compiler and so can fail in runtime – thus the unwraps.

We also shouldn’t forget that in Rust, panics across FFI boundaries are an undefined behavior. To avoid doing that, we should catch them using catch_unwind. We may then try to convert the error to a string. Regardless, if we catch a panic, we should indicate to the calling Scala code that the native code failed. A natural solution is to throw a JVM exception, using throw_new. Note that calling this function doesn’t immediately throw. You still need to finish the native function, for example by returning some value. It doesn’t matter which, because it will be discarded. Only then will the exception be thrown.

  let result = panic::catch_unwind(|| {
    // implementation goes here
  });
  result.unwrap_or_else(|e| {
    let description = e
      .downcast_ref::<String>()
      .map(|e| &**e)
      .or_else(|| e.downcast_ref::<&'static str>().copied())
      .unwrap_or("Unknown error in native code.");
    // don't `unwrap` `throw_new`, another JVM exception might have already been thrown, in which case the `Result` is `Err`
    let _ = env.throw_new("java/lang/RuntimeException", description);
    -1 // some value to satisfy type checker, it won't be used
  })Code language: Rust (rust)

All together, the sample Rust code looks like this:

// native/src/lib.rs
use std::panic;

// This is the interface to the JVM that we'll call the majority of our
// methods on.
use jni::JNIEnv;
// These objects are what you should use as arguments to your native
// function. They carry extra lifetime information to prevent them escaping
// this context and getting used after being GC'd.
use jni::objects::JObject;
use jni::sys::jint;

// This keeps Rust from "mangling" the name and making it unique for this
// crate.
#[no_mangle]
pub extern "system" fn Java_com_github_sideeffffect_scalarustinterop_Divider_divideBy(
  env: JNIEnv,
  object: JObject,
  denominator: jint,
) -> jint {
  let result = panic::catch_unwind(|| {
    println!("Hello from Rust!");
    let numerator = env.get_field(object, "numerator", "I").unwrap().i().unwrap();
    println!("Printing from rust library. numerator: {}", numerator);
    println!("Printing from rust library. denominator: {}", denominator);
    numerator / denominator
  });
  result.unwrap_or_else(|e| {
    let description = e
      .downcast_ref::<String>()
      .map(|e| &**e)
      .or_else(|| e.downcast_ref::<&'static str>().copied())
      .unwrap_or("Unknown error in native code.");
    // don't `unwrap` `throw_new`, another JVM exception might have already been thrown, in which case the `Result` is `Err`
    let _ = env.throw_new("java/lang/RuntimeException", description);
    -1 // some value to satisfy type checker, it won't be used
  })
}Code language: Rust (rust)

Conclusion

In this post, we have seen a short tutorial on how to call to native Rust code from Scala on JVM via JNI. Scala and Rust can be used together for great synergy, yet there doesn’t seem to be any documentation on how to do that – I hope this post has helped to mitigate this blind spot. While using JNI can be a bit bothersome, there are tools and libraries that help, like the Rust crate jni and Scala’s suite sbt-jni. Initially, there wasn’t support for compiling Rust in sbt-jni, but support was eventually added and I’m happy to report that it already seems to be used in the wild.

I would like to thank Michal Vaner (our Avast ex-colleague) for his help on the Rust side, particularly for making me aware of the issue with panics across FFI and instructing me how to catch the errors and convert them to a sensible string. I’m also grateful to the authors of the Rust crate jni, to Jakob for creating sbt-jni and to Grigory for taking over the maintainership.

Head over to github.com/sideeffffect/scala-rust-interop to get your hands on the code, it is executable (sbt core/run) and there are also sample tests (sbt test). I hope using Scala together with Rust becomes more popular in the future, the benefits are significant. Merry Scala and Rust hacking, everyone!