Multiplatform + native C library + WASM?

I’ve been scratching my head about this, so here goes. I’m one of many people working on an open-source project called ElectionGuard, which is a sort of end-to-end verifiable voting technology. There is currently a “reference” implementation in Python, and more limited implementations in C++, Haskell, and other languages.

To make the code performant, the bottleneck is modular exponentiation of 4000-bit numbers. GnuMP is really fast, yet can still take several CPU seconds to encrypt a single ballot. Java’s BigInteger is about half that speed. The pure-Kotlin kt-math is a third-again as fast as BigInteger. The other alternative is something called HACL, which comes out of Microsoft Research. It’s in C, but was generated by a fancy, formally verified toolchain; it’s basically the same speed as GnuMP in performance, but with a strong case that it’s bug-free. (GnuMP includes native assembly language for x86, ARM, and others. It’s never failed me in practice, but I wouldn’t be willing to bet on any absence of bugs.)

The current ElectionGuard-Python codebase uses GnuMP. The current C++ codebase uses HACL. And there are other less-complete implementations using other bignum libraries. My personal mission is to see whether we can truly write ElectionGuard, as a general-purpose library, exactly once, and be able to deploy it across a huge variety of platforms (desktop, mobile, in-browser, giant cluster) for very different use cases, all with the most performant code.

If you ask what programming language is the most likely to be able to target all these different kinds of platforms with a single codebase, then Kotlin rockets to the top. So here’s what I’m concerned about. If any of you have dealt with this before, I’d appreciate your feedback.

  • Android or desktop JVM: it would be fairly straightforward to build a JNI bridge from native methods to HACL. The HACL library rarely called malloc/free. For the most part, you just hand it arrays of uint64 as input or output and it mutates the given output array. This suggests that we could use Kotlin’s ULongArray as the underlying datatype and build just enough of a wrapper to get to the internal data, pass that pointer into the C code, and call the HACL methods. Bonus points that you can write unit tests that compare HACL against BigInteger, which should give identical answers.

  • iOS / Desktop Native: I’m assuming this is a straightforward application of Kotlin/Native for calling out to a C library.

  • Browser / WASM: One of the use cases is that you’ve got a sophisticated in-browser app, built with something like React/Native, which needs to call out to ElectionGuard. If ElectionGuard was pure Kotlin, this would be easy. Declare a JavaScript interface. Use the JS backend. Done. But we’ve got this C library! The bridge between the JS side and the WASM side is new to me, and looks like you’ve got to do a lot of copying to get data back and forth, which could be a performance issue. This suggests that I’d want to compile most of the Kotlin code (“natively”), link it against the C code, and have the backend be WASM for that, with a thin JS frontend. (Justification: when you’re doing a longer math expression using lots of these modular exponentiations, you win big if you can avoid copying 4000-bit numbers back and forth from WASM memory to JS memory.)

So, my questions is whether anybody has tried to do “multiplatform” across all these different platforms, particularly including linking against a C library. Bonus points if it’s an open-source codebase that I can steal stuff from. Conversely, I’m curious if anybody can look at this set of requirements and declare that I should give up on one of these targets, for whatever reasons, but the rest are all just fine. I’m extra curious if this will all get better/simpler with Kotlin’s native WASM generation.

Side-note: the HACL people have made their own bindings to a limited subset of the HACL library, via WASM. It’s probably a reasonable thing for me to ask them to add bignums to this. An interesting question then becomes whether Kotlin/WASM can target a precompiled WASM library like this, so you get a singular WASM image, without any need to copy back and forth from the JS layer.

You might ask how hard would it be for HACL to target something besides C or WASM? Their formal verification logic knows a lot about the semantics of C code. They have a newer backend to generate WASM, but they’ve chosen to support OCaml via bindings to the C code, rather than just generating new OCaml. I feel very safe saying that it would be radically simpler to build Kotlin/Native bindings to the C code than to get HACL to generate Kotlin code.

Edit: I took a look at Android, and their version of java.math.BigInteger turns out to use native code for everything already (apparently bottoming out at the OpenSSL library), as opposed to the OpenJDK code where it’s in pure Java. I haven’t benchmarked the performance difference, but suffice to say that there’s an argument here that you just want to use BigInteger on every Java platform and keep things simple.


We have been doing some performance tests for BigInt in KMath. Right now is more or less the same for KMath-core-jvm as Java BigInt, but we did not test it on JS.

The trick with those things is that different implementations have different strong and weak sides, so the only way to get good performance is to fine-tune each operation intrinsic for a specific problem.

It does not seem that you have any memory-intensive operations here so it should be possible to reach the same performance as in C in Kotlin. I suggest working in this direction - suggest improvements for existing libraries. Kotlin will support JVM, JS and WASM and could be fine-tuned for performance on each of those platforms.

1 Like

Update: I’ve been pounding away on this. I’ve now got HACL working with Kotlin/Native (running maybe 2/3 the speed of java.math.BigInteger on the JVM) and kt-math working with Kotlin/JS, albeit a factor of 60x slower than kt-math on the JVM. This is modestly surprising. But at least it works, and all my tests pass.

I made an attempt at using SJCL (Stanford JavaScript Crypto Library). Dukat kinda imported the TypeScript definitions, but I had to make a wide variety of changes to get it to compile, and I’d run into a brick wall where I’d get “undefined” in the middle of a computation, for no apparent reason.

I also made an attempt at using gmp-wasm, which uses WebAssembly, but everything there uses Promise-based APIs. So far as I can tell, I’d have to rewrite my entire library to use suspend functions, since JavaScript (so far as I can tell) doesn’t let you use runBlocking or whatever to just wait for an answer to come back.

Anyway, I’m a bit frustrated by all this. Any thoughts on how to make this multiplatform stuff really work well?

Hi, I am working on a project on my own that I try to run on Kotlin/Multiplatform.
Currently I am developing a bridge between Kotlin/JVM to C via JNI.
I have gotten the C library to compile using the cpp-library grade plugin, I also managed to copy the .so/.dylib/.dll libraries into the Kotlin/JVM part of the Multiplatform library.

My current challenge is to export the Multiplatform library to another Multiplatform library and being able to make JVM to find the native C/JNI libraries, it’s not in the runtime paths or something similar, I guess it is a kotlin-dsl/gradle related configuration problem.