How to Use the Foreign Function API in Java 22 to Call C Libraries

> I am sort of surprised that there isn't a widely used tool that uses codegen to generate jni bindings sort of like what the jna does but at build time

There are several, including SWIG.

There is SWIG, which does bings to and from C for almost every language that exists.

Back in 1998 I wrote a code generator to make JNI stubs for LAPACK. It’s the kind of programming that goes that way.

There is a library called jnigen [1], mainly used by the libGDX framework [2]. But I don't see it used in many other projects though. Personally I use it to maintain a set of Lua C API bindings for some platforms [3] and it works sort of OK once you manage to somehow set up a workflow for building and testing the binaries.

> It works but it is a pain if anything ever changes or a new platform needs to be brought up. Checking in binaries always feels icky but is necessary if not all targets can be built on a single machine.

It is definitely a pain when you cannot test all changes on a single local machine. But I would argue that it is true whenever multiple platforms (or maybe even multiple glibc versions) are involved, regardless of what languages/libraries/tools you use.

[1] https://libgdx.com/wiki/utils/jnigen [2] https://github.com/libgdx/libgdx [3] https://github.com/gudzpoz/luajava

Native libraries are typically packaged inside a jar so that everything works over the existing build and dependency management systems.

For example, each these jars named "native-$os-$arch.jar" contain a .dll/.so/.dylib: https://repo1.maven.org/maven2/com/aayushatharva/brotli4j/

JNA will extract the appropriate native library (using os.name and os.arch system properties), save the library to a temp file, then load it.

> Every ffi example I've found seem to operate on the assumption that you want to invoke syscalls or libc ... Java already mostly has decent facilities to interact with even without native calls.

Because you would use ffi to interact with libraries that don't have Java wrappers yet: IE, you're writing the wrapper.

Using syscalls or libc is a way to write an example against a known library that you're probably familiar with.

The recommended distribution model for Java applications is a jlinked runtime image [1], which supports including native libraries in the image.

[1]: Technically, this is the only distribution model because all Java runtimes as of JDK 9 are created with jlink, including the runtime included in the JDK (which many people use as-is), but I mean a custom runtime packaged with the application.

So, other people have already answered this, but this does seem to be a gap where many developers lack some piece of knowledge to chain the whole solution together. You normally package this sort of thing by putting the native library in a jar, extracting it to a tmp file that will be deleted on exit, and opening that dynamic library.

I’ve met many perfectly reasonable developers who do know all those steps can be done but can’t put them all together - maybe because it just hasn’t clicked that you can store a library in a jar. It feels like something tutorials should cover, but I think falls into the, “surely everyone can work it out?” category.

If your app is open source, or you're willing to buy a commercial tool, then you could try Conveyor from my company [1]. It will:

- Find all the shared libraries in your JARs or configured app inputs (files in your build/source tree)

- Sniff them to figure out what OS and CPU arch they are for

- Bundle them into the right package for each platform that it makes, in the right place to be found by System.loadLibrary()

- Sign them if necessary

- Delete them from the JARs now they are extracted. Optionally extract them from library JARs, sign them and then put them back if your library refuses to load the shared library from disk instead of unpacking it (most libs don't need this)

- JLink a bundled JVM for your app for each platform you target, using jdeps to figure out the right set of modules, and combine that with your shared libs.

When building Debian/Ubuntu packages it will also:

- Read the .so library dependencies, look up the packages that contain those other shared libraries and add package dependencies on those packages, so "apt install" will do the right thing.

So that makes it a lot easier to distribute Java apps that use native code.

[1] https://www.hydraulic.dev/

You do it the standard way, package them inside the jar file.

Put them in a jar?

Java's FFI is currently a very low-level. As the article points you, you don't actually have to do this: the jextract tool will generate the bindings for you from header files.

I'm sure someone will come along and write annotations to do exactly as you describe there. The Java language folks tend to be very conservative about putting stuff in the official API, cuz they know it'll have to stay there for 30+ years. They prefer to let the community write something like annotations over low-level APIs.

Anyway, the GraalVM folks don't have quite the same limitations as Java, so they have annotations already (https://yyhh.org/blog/2021/02/writing-c-code-in-javaclojure-...):

    @CStruct("MDB_val")
    public interface MDB_val extends PointerBase {

        @CField("mv_size")
        long get_mv_size();

        @CField("mv_size")
        void set_mv_size(long value);

        @CField("mv_data")
        VoidPointer get_mv_data();

        @CField("mv_data")
        void set_mv_data(VoidPointer value);
    }

Can be even simpler now (you can declare it as a local function in a method, so this works when copied to Program.cs as is):

    var text = "Hello, World!"u8;
    write(1, text, text.Length);

    [DllImport("libc")]
    static extern nint write(nint fd, ReadOnlySpan<byte> buf, nint count);

(note: it's recommended to use [LibraryImport] instead for p/invoke declarations that require marshalling as it does not require JIT/runtime marshalling but just generates (better) p/invoke stub at build time)

Yep, that is my main complaint, and why I will rather reach to JNI instead.

I have played with raylib bindings for clojure by using the new foreign function api. It was a lot of fun. SDL might be a better fit because it prefers pass by reference arguments [0].

[0] https://gist.github.com/raysan5/17392498d40e2cb281f5d09c0a4b...

It seems it will make it somewhat easier.

But if you want to use SDL2 from something higher-level, you will be much better served by C# which will give you minimal FFI cost and most data structures you want to express in C as-is.

Oh man that's a cool idea.

Might just build a SDL2-wrapper for ffi just as an FFI and FMI-exercise.

The answer is basically you don't. It's impossible to make a sane, stable FFI for a language unless you put it behind a C ABI, which is relatively basic, but this is exactly why it's most suitable for FFI: implementing support for calling C functions is way more trivial than figuring out how to call the latest C++/Rust/etc monstrosity.

You put your C++ behind a C API.

This is something new. Before it you had to create a native-compatible shared library that returns jString/jObject instead or use a proxy which did this for you (JNA). Let's see what happens next, maybe even shiboken

I don't know why people don't know this, but you can just use GObject.

There's javacpp which can do that.

For the heap, JEP 454 is reasonably detailed: https://openjdk.org/jeps/454

It describes how to adopt memory from C and have C adopt memory you allocate, and gives control over how memory is allocated in an arena.

The arena has lifecycle boundaries, and allocations determine the memory space available. Java guarantees (only) that you can't use unallocated memory or memory outside the arena, and if you access via a (correct) value layout, you should be able to navigate structure correctly.

The interesting stuff is passing function pointers back and forth - look for `downcall method handles`.

Don't have an article, but the gist on stacks is that Java still uses the regular architecture stack (rsp on x86, etc) that the FFI'd code will, and on exit to/entry from FFI it'd have to store its stack end/start pointer (or otherwise be able to figure the range out) such that GC knows what to scan.

I would give the jextract tool a try. I believe it uses LLVM to parse the header files, so the generated bindings might actually be pretty good.

Invoking native code has always been necessary in Java. In the past it was done via JNI which has many issues. These new APIs solve the issues and simplify the API. The use case is interacting with anything that isn't written in Java.

Same use case as to why .NET has low/zero-cost FFI.

This is similar, except more boilerplate and much, much slower.

Memory barriers don't force a flush of all CPU cache. They will enforce the ordering of memory operations issued before and after the barrier instruction, preserving the contents of the CPU's various caches.