Why Kotlin Native?

@JanKanis - Look at this post for more insight into what target platforms/markets JetBrains might be targeting with Kotlin Native.

Following the same logic, we should have fixed existing IntelliJ IDEA problems before starting the work on ReSharper, RubyMine or Kotlin - and because there are always existing IntelliJ IDEA problems, these projects would never have started.

To reiterate, a separate team works on Kotlin Native; none of the resources of the existing Kotlin team were diverted to that project.

6 Likes

I’m a perfectionist, I appreciate that’s not always (or even often) practical in the real world. Nevertheless, I think there’s something to be said for structuring work such that most issue can be ironed out before launching in other ventures. It forces you to simplify.

Anyhow, don’t let my barb lower morale. I’m certainly excited for Kotlin Native!

1 Like

I’m also a little skeptical, but as JetBrains had a hiring opportunity that meant native vs Java vs JavaScript wasn’t zero-sum, that does eliminate the worries over the existing userbase being deprioritised.

The big looming problems I can see now, without any obvious solutions:

  • Yes in some senses Kotlin is not just a better Java, but Kotlin’s standard library is basically Java’s standard library + some thin extensions, so in other senses it is. All existing Kotlin code is full of import java.stuff, and I’m guessing 95%+ also imports open source Java libraries. Kotlin Native could perhaps reimplement some of the Java standard library, but out of the box, if you grab some arbitrary Kotlin code it isn’t going to run on Kotlin Native unless there’s an AOT Java compiler included. Same is true for Kotlin JS of course but the limited browser environment means people expect an anemic standard library anyway.
  • “Native” often implies “no GC”, especially in the embedded realm. All existing Kotlin code expects GC to be present. Perhaps Kotlin Native will still be a GCd language. If it’s not, then at that point without a GC and without the Java standard library, they’ll basically be two separate languages.

So it’ll be interesting to see what design decisions JB makes here.

By the way, just as a reminder, you can combine Kotlin and “small standalone native binaries” today. Look at the Avian examples:

https://readytalk.github.io/avian/

One of the projects I keep trying to find time for is a tool to make creating such binaries brain-dead simple.

1 Like

I would be really surprised if they’d come out with a Kotlin native compiler that doesn’t also compile Java. That would just split the ecosystem for no reason, especially as Java is basically a subset of Kotlin in semantics. I wouldn’t be surprised if it will just take class files as input. Andrey already said they were going to implement a GC.

My interpretation of what Jetbrains people have said about Kotlin Native is that they think IoT/embedded is going to grow a lot in the future, which is probably true. These devices are getting plentiful and powerful enough that bare metal languages (C/C++/D/Rust) aren’t a necessity anymore and higher level languages make sense, but not powerful enough to run a full JVM. There aren’t many well known languages in this space of high level + native compiled. What I can think of there’d be Go, Java with existing AOT compilers, or native compiled .Net, and Go is still a much smaller ecosystem than the other two. I think Jetbrains thinks they can create a good enough alternative with Kotlin Native to have an impact. I’m not very familiar with the state of existing Java AOT compilers, but I wouldn’t be too surprised if Jetbrains thinks that existing tools are not good enough and it will be easier to create a new compiler based on LLVM. And a Java-ecosystem based language would be uniquely suited for that since it is the only major ecosystem that is mostly self contained, in the sense that it is very rare for Java apps and libraries to have mandatory dependencies on native libraries. Which is unlike the .Net, Go, and any other open source language ecosystem.

A Kotlin Native offering would offer the same selling points as Kotlin/JVM but for the IoT and embedded market. I.e. a language that is very easy to learn for lots of developers and a familiar environment with a huge library ecosystem. Splitting the Kotlin ecosystem by not compiling Java libs would be a bit nonsensical, since then you would just have a new language and a new ecosystem with few libraries, which is not a very attractive proposition to a prospecting IoT developer.

The only thing I’m missing is how exactly Jetbrains plans to profit from this. Selling more tools to IoT devs is nice, but maybe they’re also planning to be an IoT or Kotlin consulting company or something like that.

2 Likes

I agree with @JanKanis, and disagree with my teammates from JetBrains that we need to compile Kotlin directly to LLVM. Unfortunately, I could not push my opinion to my teammates, so it remains my personal opinion.

Kotlin is an awesome languages for JVM, created WRT good interop with Java and with tons of Java libraries. Indeed, there are some disadvantages in JVM, at least, in OpenJDK, like large runtime size, JIT (as opposed to AOT) which can be inacceptable in some cases, unpredictable GC pauses, etc. That does not mean, however, that we should compile Kotlin ourselves directly to LLVM. This means that we need more appropriate JVM implementation. There are some, like RoboVM, MOE, etc. We can pick one and contribute our code there. We can even invent our own JVM AOT, it’s not that hard, at least comparing to Kotlin Native which has to implement semantics which is close to JVM. I see some advantages there:

  • We don’t have to do large amout of work, we can easily start with what we have and then make things better.
  • We could get a good adoption from people who have experience with Java and a little unsure to start using Kotlin. With JVM AOT approach they could write most of their code in Java and start to write some of the code in Kotlin (and fall in love with it). With current approach these people either refuse to port their app to iOS at all or discover and use MOE.

Let me tell about some myths about “ugly monstrous JVM”

  1. JDK is not necessary large. It is, but we can cut only parts necessary for embedded/AOT/iOS/games/whatever else, and it can be small enough. We can even use minifier like proguard or write our own one, which is capable of eliminating unused classes and methods.
  2. JVM is not necessarily JIT. We can sacrifice some ClassLoader features to have chance of creating pure AOT. AFAIK, RoboVM does not support class loaders, and it’s not a big limitation for iOS and embedded development.
  3. JVM implementation is not tied to GC, any automatic memory management mechanism is acceptable, including ARC, region-based memory management, etc.
  4. JVM is heavyweight. JVM bytecode consists of very simple instructions that remind of assembler. Full spec describes about 200 instruction versus x86 architecture which has thousands of instructions. Most of instructions are like iadd, fadd, isub, fsub, etc, which are trivially mapped to LLVM IR.
  5. GC does not necessarily produces unpredictable pauses, there are GC scheduling algorithms that give strong guarantees of upper limit of time spent in GC.
5 Likes

So @Alexey.Andreev you’re saying Kotlin Native will only accept Kotlin code as input and not Java? I’ll be sitting here being very surprised then. And what’s the story for the standard class libraries and e.g. Android framework code?

This will require licensing from Oracle and formal confirmation of full Java SE compatibility test suite. Otherwise they will sue JetBrains to NPE. They skipped RoboVM and other minor implementations due to insignificance and short life time.

Tell this to guys from excelsior. With stable implementation for x86, x64 and arm took like 4 years.

This is directly forbidden by licensing. This kind of cuts are formal reason for Oracle - Google ongoing trial.

2 Likes

This will require licensing from Oracle and formal confirmation of full Java SE compatibility test suite. Otherwise they will sue JetBrains to NPE. They skipped RoboVM and other minor implementations due to insignificance and short life time.

I’m not sure why this require licensing. Calling this implementation “Java SE” implementation will require passing JCK. Can Oracle sue product not tagged as “Java SE”? If can, is this so hard to pass JCK?

Tell this to guys from excelsior. With stable implementation for x86, x64 and arm took like 4 years.

I’m not sure how they actually implemented their AOT compiler. Did they write codegen themselves or they used LLVM? In the first case they had to create a good codegen, and creating a good efficient codegen is very complicated task. Also, they support class loading, therefore they have JIT as well. Creating subset of JDK (say, without ability to create new class loaders) won’t require JIT, which reduces amout of work. Also, did they implement JDK classes themself? If they used forked OpenJDK and licensed their product under LGPL, they also could reduce amount of work. Also, they support threads and memory model, which is quite complicated, even with LLVM. I believe, it’s possible to create something like green threads upon runtime coroutines.

This is directly forbidden by licensing. This kind of cuts are formal reason for Oracle - Google ongoing trial.

Can you provide a proof link? If this is the case, can they sue for automatic tool that takes only part of JDK and compiles it to native code? I guess it’s all about distribution, I don’t say about distributing cut version of JDK.

There is another issue. If you want to pass the JCK you will effectively have to build a JVM. The JVM is a complex beast and the Java standard library is tightly linked with it. Looking at it, the biggest paintpoints would probably be classloaders (the need to be able to load class files on demand), reflection (but Kotlin has its own reflection API anyway) and JNI.

A big win for native would be reasonably straightforward interop with existing native libraries, which means it needs to have a way to interact with C-style API’s and perhaps even C++ (perhaps with some limitations), but at least non-gc languages.

If you want to use the full power of the java standard library, why not stick with the JVM? Long term I see native as both native kotlin replacements and native exising library interop. Doing that would allow the advantages of being able to use the same code base for various platforms without having the tight semantics (and the hefty cost) of trying to stay very close to the JVM. It would also be in line with the JS target that does not stick too close to JVM semantics either.

1 Like

Anybody can sue for anything. Thats how courts work. If Kotlin becomes big enough that is just inevitable. The main difference is Oracle being very aggressive with Java copyrights and patents lately. Compare this say to Microsoft with Mono which they basically just bought. Making Java compat shims is like touching lava right now. It definitely requires serious legal analysis.

It is GPL with linking exception, not LGPL. It is very viral license and will require some tricky solutions to stay on apache license if you want to include compilation results into single binary. For example AOT compilation with static linking with GPLed OpenJDK may transfer GPL to compiled end user code. Maybe. Depends. I never read OpenJDK linking exception clause.

Google dodged bullet from Oracle by claiming fair use of copyrighted API. This means software API’s are copyrightable as seen from USA courts. Current appeal by Oracle hopefully will change this.
More details:
Google LLC v. Oracle America, Inc. - Wikipedia.

I agree with @pdvrieze that Kotlin will need to interop with whatever libraries/frameworks are present on the platform that Kotlin Native is targeting. Trying to take the JVM ecosystem to every platform isn’t feasible since the ecosystem itself is tightly coupled to the JVM. There might be even be some serious legal issues that could arise from the Oracle vs Google case that is currently ongoing.

Take the Web platform for example where there are many useful libraries/frameworks that Kotlin developers WILL want to take advantage of. Any programming language that wants to target the Web (front-end, and possibly back-end) needs to seamlessly interop with the existing libraries/frameworks there. When it comes to memory management @Alexey.Andreev might have a point with Kotlin using something like ARC (Swift uses this system), instead of a GC.

Is ARC really a form of automatic memory management?

2 Likes

Yes, ARC is technically a form of automatic memory management but suffers from the drawback that it is unable to deal automatically with cyclic references. You therefore have to deal with these manually using weak references which is not always straightforward.

However, I believe JetBrains have already said that they’re building their own ‘native friendly’ GC for Kotlin Native. I don’t know whether this means that you’ll be able to suspend the GC when executing a critical piece of code, optionally use manual memory management or both. We’ll just have to wait and see.

Looking at this outdated document there are some other things that might be covered by Kotlin Native. See that native interop is being considered. Very interesting that Kotlin Native could have its own concurrency layer.

1 Like

I think Kotin Native makes sense. Golang has been a hit because of channels, coroutines and it allows you to produce a single executable file that’s easy to deploy. Kotlin will have all of that plus decent syntax.

If enough of the basic JVM libraries can be rewritten or replaced with Kotlin libraries, Kotlin may get an added boost.

Jetbrains has managed to produce a very nice language with relatively little investment. If native works as well as JVM, then it could be great.

1 Like

Wow! Amazing!
I expect Kotlin Native to beat Rust which is really a headache.
Is there somewhere I can track the status of this project?

3 Likes

Presumably the drive behind Kotlin native is to move the JetBrains IDEs off of the JVM.

Rust is designed as a much lower level language with e.g. semi-manual resource management. Kotlin (unless Kotlin Native is going to be very different from Kotlin-JVM in this regard) is a much higher level language. If you don’t need the speed & control Rust gives you something like Kotlin will likely be more suited to a task.

There’s no official release or announcement of Kotlin Native yet, we’re still waiting for that.

Unlikely. One of their reasons for creating Kotlin was to have something interoperable with Java because rewriting all of their code was not an option. And what advantage would a native IDE give over a JVM based one?

1 Like

When Kotlin Native eventually emerges, I wouldn’t be surprised to see plug-ins for AppCode (for iOS development) and CLion for everything else. These might be a more natural home for it than IntelliJ IDEA and would help to popularize the aforementioned IDEs.

I tried Rust myself but soon gave up. I found that my thinking was dominated by memory management and consequently it was difficult to concentrate on what I was actually trying to program!

1 Like