Interop with Google's J2CL - what would it take?

JavaScript
1

Wouldn’t it be cool if you could take a mixed Java+Kotlin codebase and transpile it to JavaScript?

Perhaps one option is interoperability with Google’s GWT successor, J2CL (in the process of being open-sourced now, AFAIK).

J2CL, in a nutshell (more info here: https://www.youtube.com/watch?v=P4VhPck5s_g after 36th minute)

  • Fast incremental transpilation: one Java file to one readable ES6 JS file
  • No optimizations. Minifying, dead-code elimination, even transpiling to ES5/ES3 are all expected to be done by standard JS tools like Google Closure and perhaps Webpack in future
  • Some small tricks like name mangling for overloaded methods and Java “this” semantics still in place
  • Minimal Java JRE (shared with GWT and j2objc) for supporting Java collections and utility classes - also transpiled to JS
  • jsinterop (https://docs.google.com/document/d/10fmlEYIHcyead_4R1S5wKGs1t2I7Fnp_PaNaa7XTEk0/edit#) - a spec & collection of Java annotations for interoperability with JavaScript. A way to “consume” existing JavaScript libraries as well as to export Java code for consumption from JS
  • jsinterop-generator (https://github.com/google/jsinterop-generator) - a project able to parse Closure-extern files and generate jsinterop-annotated Java interfaces for consuming existing JavaScript libraries. Might support ts.d files in future as well

I think philosophically, Google recognizes that nowadays languages other than JavaScript are all niche players in the contemporary JavaScript stack, so they need to interoperate very well with it rather than replace it (as GWT tried) with an “end all solution” that falls under its own weight as nobody has the time to maintain it.

(… not that they had any other option in 2006, when Webpack, TypeScript, React etc. did not even exist - but the world is quite different in 2017.)

To make the interoperability between Java and Kotlin as smooth as possible, Kotlin and J2CL must not only interoperate on the jsinterop level - a closer integration might be necessary, where I am not supposed to annotate a Java class with jsinterop annotations so as to use it from within Kotlin. That probably means that J2CL and Kotlin have to somehow “agree” on using common name mangling scheme when transpiling overloaded methods to ES6. I’m sure I’m missing a lot of additional details here.

Thoughts?

2 Likes
2

teavm is capable of this right now.
author is working at kotlin team btw.

3

Where’s this J2CL? I heard rumors that “j2cl is a future of GWT”, but saw no tool to play with, nor any documentation.

I don’t believe in possibility of doing good DCE in JavaScript. It’s too dynamic to be statically analyzed, which is required by DCE. Google Closure Compiler works with subset of JavaScript which is easy to analyze, it even does not support Object.defineProperty, widely used by Kotlin to JS compiler.

As for Webpack, yes, it does some sort of DCE on module level. This requires you to split your code into small modules. In case of compilation single Java or Kotlin source file to a single JS module, this would get somewhat similar to DCE. However, this makeshift DCE can’t eliminate unused methods from within classes. Also, note that currently Kotlin compiler does not support file-to-file compilation, which is going to make some problems interacting with J2CL toolchain.

That’s going to be a problem - Kotlin goes with its own implementation of Java-like collections, it’s a little unclear how it should interact with J2CL. Should it drop its own collections and, like JVM compiler does, map them to J2CL collections? Should it be compiler option for particular use case? What about backward compatibility?

Kotlin already has its own interop with JS, which is more powerful. For example, Kotlin has concept of package-level function which maps directly to JavaScript functions, In JSInterop you end up with static methods of a class, which looks unnatural for Kotlin. I guess, to effectively interact with J2CL Kotlin has to fallback to limited interop, compatible with JSInterop.

4

Interoperation between 3 languages? Java, JavaScript and Kotlin. As you already said, Java is a niche player in JavaScript stack. So, in JavaScript stack most users would like to interact with JavaScript rather than with Java. There are few users who want Java (GWT, J2CL) in their JS apps, and only few of them who needs Java and Kotlin in a single JS-targeted project. Can you name possible use cases?

5

There are few users who want Java (GWT, J2CL) in their JS apps, and only few of them who needs Java and Kotlin in a single JS-targeted project. Can you name possible use cases?

Most Important Point: If you don’t believe that the people who might want to transpile Java and Kotlin to JS are a sizable community, there is no point in discussing anymore, isn’t it?

Yet a significant reason why Kotlin is successful - I would assume - is because of (a) its perfect interoperability with Java and because (b) it can be introduced in a Java project incrementally. If I have a bunch of shared Java classes that run on the server, and are transpiled to JavaScript, what are my chances of introducing Kotlin into the mix? Rewrite my whole frontend in Kotlin? Really?

As for not being capable of doing a good DCE in JavaScript - you might have the perfect DCE in Kotlin, but nobody would use it. Because e.g. I would like my DCE to optimize not only my tiny Kotlin 10K LOC codebase, but also the gazillions of React, ReactNative, you-name-it 3rd party JavaScript libs which Yarn had downloaded and called from within my Kotlin codebase. Perhaps its my ignorance - how is that problem solved in Kotlin? BTW GWT (with its own DCE) has no solution, hence one of the reasons J2CL is pushed.

Regarding J2CL nowhere to be seen. Yes, that’s a problem - and it hurts. :slight_smile: But I thought - perhaps it is better to start brainstorming early, before decisions are taken and bridges are burned. :slight_smile:

For the other challenges, like unifying the JREs and the interops. I have no easy solutions as I’m no expert. I guess if there is a will, there is a way. Which goes back to the “Most Important Point” from above.

1 Like
6

Ok, I see. Right for this use case I created my own project, called TeaVM. The only drawback: it reinvents entire JavaScript stack, including DCE and minification. While there’s no solution for Kotlin JS + J2CL (since there’s no interop between them, not even the second one being available somewhere). You can give it a try, as well as another similar tool like bck2brwsr or dragome. I guess you answer “but they don’t interact well enough with modern JavaScript stack”. But why don’t you try, may be for your particular project their interaction will be good enough? IMO, it would be much better than waiting until J2CL gets published an Kotlin team implements J2CL/Kotlin interop.

I would like my DCE to optimize not only my tiny Kotlin 10K LOC codebase, but also the gazillions of React, ReactNative, you-name-it 3rd party JavaScript libs which Yarn had downloaded and called from within my Kotlin codebase

You are right, except for there’s no problem to apply Kotlin DCE to Kotlin code and JavaScript DCE (say, Webpack/Rollup/whatever else) to your JavaScript codebase. The only problem here is you want to create Java/Kotlin clases to be called from JavaScript. I mean, not thouse which are passed from Java/Kotlin to JavaScript and then called back, but those which are directly instantiated from JavaScript. According to my experience it’s a rare case.

Perhaps its my ignorance - how is that problem solved in Kotlin?

Right now we are working on it. We develop our own tool that performs DCE in JavaScript generated by Kotlin code, and if you want some Kotlin classes or functions be directly called from JavaScript, you should mention them directly in ‘keep’ list. Also, we are going to implement file-to-file translation eventually, but it’s going to be a little challenging for now. And Kotlin does not support Google Closure Compiler (and I’m not sure it’s even possible without patching GCC, this requires additional investigation). But remember, GCC is a very specific tool which does not work with any JavaScript framework, only with a JavaScript framework which supports GCC.

7

If I have a bunch of shared Java classes that run on the server, and are transpiled to JavaScript, what are my chances of introducing Kotlin into the mix? Rewrite my whole frontend in Kotlin? Really?

Ok, I see. Right for this use case I created my own project, called TeaVM. The only drawback: it reinvents entire JavaScript stack, including DCE and minification. While there’s no solution for Kotlin JS + J2CL (since there’s no interop between them, not even the second one being available somewhere). You can give it a try, as well as another similar tool like bck2brwsr or dragome. I guess you answer “but they don’t interact well enough with modern JavaScript stack”. But why don’t you try, may be for your particular project their interaction will be good enough? IMO, it would be much better than waiting until J2CL gets published an Kotlin team implements J2CL/Kotlin interop.

We are not in a hurry. :slight_smile: The whole interop topic is a long term thing isn’t it?
The codebases that already transpile Java-to-JavaScript are usually bound to GWT. Migrating from GWT to teavm just for the option to add a bit of a Kotlin to the mix is a huge work with a risky proposition. As much as I admire your teavm project (and Tulach’s bck2brwsr efforts) - they are a one-man free-time shows (sorry - I had to name the elephant in the room!) so - no go - unfortunately. Going with a J2CL+Kotlin combo (if this ever sees the light of the day) would be a completely different thing, for obvious reasons.

I would like my DCE to optimize not only my tiny Kotlin 10K LOC codebase, but also the gazillions of React, ReactNative, you-name-it 3rd party JavaScript libs which Yarn had downloaded and called from within my Kotlin codebase

You are right, except for there’s no problem to apply Kotlin DCE to Kotlin code and JavaScript DCE (say, Webpack/Rollup/whatever else) to your JavaScript codebase. The only problem here is you want to create Java/Kotlin clases to be called from JavaScript. I mean, not thouse which are passed from Java/Kotlin to JavaScript and then called back, but those which are directly instantiated from JavaScript. According to my experience it’s a rare case.

Not necessaruly a rare case. E.g. React relies on being able to construct your Kotlin/Java component from within JavaScript, and to call methods on it which have certain fixed JavaScript names from within JavaScript. That’s what GWT/JSInterop is about - “freezing” those things so that they are not mangled/minifed by GWT’s DCE.

Right now we are working on it. We develop our own tool that performs DCE in JavaScript generated by Kotlin code, and if you want some Kotlin classes or functions be directly called from JavaScript, you should mention them directly in ‘keep’ list.

Your ‘keep’ list sounds like the the GWT/J2CL JSInterop thing again. Have you looked into it? It is in a small JAR file that lives separately from GWT, is not dependent on GWT/J2CL and is not using the GWT namespace. Just a bunch of annotations with attached semantics. Why can’t JSweet, TeaVM and - yeah - Kotlin all agree on using the same jsinterop spec as GWT/J2CL? If nothing else, that might higher up the chances of somebody trying out these alternative solutions.

And Kotlin does not support Google Closure Compiler (and I’m not sure it’s even possible without patching GCC, this requires additional investigation). But remember, GCC is a very specific tool which does not work with any JavaScript framework, only with a JavaScript framework which supports GCC.

Really? So if I feed GCC with a closure-ified JavaScript + my pure JS React-and-whatnot dependencies, GCC would refuse to compile that input? Or will it still minify the thing, but will just not do any dead-code elimination in the pure-JS bits? Which is probably not the end of the world? Just asking…

1 Like
8

As much as I admire your teavm project (and Tulach’s bck2brwsr efforts) - they are a one-man free-time shows (sorry - I had to name the elephant in the room!) so - no go - unfortunately.

That’s a big challenge for such projects. If just anybody started using it, there would be growing community around it. But nobody is going, since it’s a single-person project. I wonder how pet projects, like Webpack or vue.js, became popular then?

And well, how did JSweet start? Was it a start-up? Where did they get money? According to their github, it’s another single-person pet project.

Not necessaruly a rare case. E.g. React relies on being able to construct your Kotlin/Java component from within JavaScript, and to call methods on it which have certain fixed JavaScript names from within JavaScript.

Do you still have to register these classes in react? If so, please, read my message carefully. I mentioned those “call-back” classes. I think there’s no problem for DCE to handle this case.

Your ‘keep’ list sounds like the the GWT/J2CL JSInterop thing again.

No, I told about DCE, JSInterop is about interop.

Have you looked into it?

Sure! From the very beginning, when it was announced in upcoming GWT 2.8.

Why can’t JSweet, TeaVM and - yeah - Kotlin all agree on using the same jsinterop spec as GWT/J2CL?

The only reason is: developers don’t have time or aren’t interested in it.

If nothing else, that might higher up the chances of somebody trying out these alternative solutions.

You just refused from trying TeaVM. If I right now thrown-away my roadmap and started implementing JSInterop, would it make you to choose TeaVM?

9

As much as I admire your teavm project (and Tulach’s bck2brwsr efforts) - they are a one-man free-time shows (sorry - I had to name the elephant in the room!) so - no go - unfortunately.
That’s a big challenge for such projects. If just anybody started using it, there would be growing community around it. But nobody is going, since it’s a single-person project. I wonder how pet projects, like Webpack or vue.js, became popular then?

I guess that’s because they solve an important use-case in a way, which is appealing to a large community? :slight_smile:

Do you still have to register these classes in react? If so, please, read my message carefully. I mentioned those “call-back” classes. I think there’s no problem for DCE to handle this case.

If a JS constructor function is a “call-back” class for you, that might solve it. Not sure honestly. And then what about the second part, where method names have to be preserved for these to be callable from React?

Your ‘keep’ list sounds like the the GWT/J2CL JSInterop thing again.

No, I told about DCE, JSInterop is about interop.

Isn’t JSInterop about both? (a) about how you map/call JS code from within Java, and (b) how your Java code naming is preserved (not minified / not removed) when it goes through the GWT (or your language-specific) DCE, so that e.g. React can call specific methods on your Java React component by assuming that you have followed the documented React naming convention for those. Duck typing.

You just refused from trying TeaVM. If I right now thrown-away my roadmap and started implementing JSInterop, would it make you to choose TeaVM?

If we decide to do the unthinkable and port our GWT codebase to teavm, having teavm supporting the JSInterop spec means we don’t have to redo all our JavaScript-facing endpoints the “teavm way”.
What do you think - is it a fad of mine, or a real usecase?

OK - seriously - we probably still won’t do it, because of other risks, but you just asked what is making a one-man project successful? That’s one thing to make it successful. Compatibility with the big players which lowers the risks & efforts for somebody willing to try your thing on a real codebase. Look at TypeScript - you can incrementally start using it, and you can mix it with JS code. And you can revert to JS. In fact - look at Kotlin on the JVM… :slight_smile:

Anyway, I stop now. Useful discussion nevertheless - thanks a lot!
Hopefully, it would influence a bit your decisions regarding Kotlin-on-JS.

Cheers, I.

1 Like
10

It’s not released yet, but will be soon. It is the technology behind Google Inbox and Google Docs.

Closure Compiler supports Object.defineProperty() and ES6 getters, these are widely used by J2CL to transpile lazy static Java initializers and JsInterop @JsProperty methods. Also, despite the dynamism of JS, Closure Compiler still beats all other tools on the market for mininizing JS, even in SIMPLE mode which works with mostly dynamic JS.

However, in advanced mode, CC does amazing DCE, moreover, CC supports deferred compilation modules and cross-module code motion optimizations. This is pretty critical in making efficient apps on the Web, for example, photos.google.com makes heavy use of this capability, in that every UI component and UI controller is a separate compilation module, and code is hoisted across dynamically loaded modules as needed on a per-function level.

I would view Closure Compiler like LLVM. J2CL, TypeScript, and other frontends lower their inputs to Closure format, which is then optimized.

1 Like
11

JsInterop really isn’t about DCE. It’s mostly about externing and lowering the impedance mismatch between JS and Java.

Really, JsInterop isn’t much different than C++ externs. C++ implements internal mangling to deal with overloading and generics. J2CL and GWT must do the same. JsInterop provides a way to get consistent naming and semantics at Java<->JS boundaries.

So for example, @JsFunction annotation allows raw JS functions passed from JS to Java to implement interfaces, e.g.

@JsType(namespace = “blah”)
class Foo {
public static void exec(SomeCallback cb) { cb.run(); }
}

@JsFunction
interface SomeCallback {
void run();
}

From JS:
blah.Foo.run(function() { console.log(‘hello’); });

It’s designed to be efficiently optimized by CC, but obey Java semantics as much as possible to avoid magic as much as possible.

12

Strange. I tried it with Kotlin in advanced mode, and it renamed property on use-site, despite the property was clearly defined via Object.defineProperty. I found the unresolved issue on GCC github, and concluded that Object.defineProperty is unsupported.

Right, it’s best among other JS minifiers, however, I don’t believe it can be better than other DCE tools built upon statically typed low-level IR.

The difference is LLVM IR is statically typed. It has notion of structures and function. In JavaScript anything can be function. There’s no unique name for a function. When you have a call site like foo() it can mean almost everything? What is foo? Where’s is defined. LLVM, WebAssembly, Java bytecode, all of them answer this question clearly, but not JS.

Anyway, currently, GCC can’t eliminate any function from Kotlin stdlib, and I see no clear reason. Is there any way to “debug” GCC? Are there any publications about algorithms used by GCC internally?

13

Oops, I was wrong, this appears to be a J2CL specific pass right now (https://github.com/google/closure-compiler/blob/master/src/com/google/javascript/jscomp/J2clPropertyInlinerPass.java)

Yes, it is more difficult to do this in dynamically typed languages, but Closure does include a type inferencer that uses abstract interpretation. If Kotlin did want interoperate with Closure, they’d have to have a mode, say, “Kotlin2CL” which compiles to JS and annotates their JS with JsDoc declarations from the Kotlin type system. Then Closure could do a much better job on the output.

The biggest problem with Web frameworks today is large monolithic single page apps, which are really bad for mobile, and so removing unused code is a problem all tools have to address eventually.

14

js can be “executed” to find code paths. See https://prepack.io/. It actually works for kotlin helloworld level project and uglified output is smaller then gcc one. More complex stuff fails though.
From my understanding reason is dom api is not fully emulated and this somehow makes emulator to stack overflow due to infinite loops. All this ?./:?/!! compiled to branches are not making its life easier too.

15

One thing I wanted to say is that this is not an all-or-nothing situation, just like WebAssembly vs Regular JS isn’t. Closure Compiler code is a typed IR. You can supply the compiler with fully typed JS, or supply it with partially typed JS. Like you could have a very large fully typed Kotlin codebase generated as Closure annotated JS code (or maybe TypeScript), and have it interface with third party JS libraries from the node ecosystem using Closure externs of TypeScript headers. This code can then be compiled together, and you’ll get good DCE on the Kotlin chunks, and with the more dynamic third party libraries opted out.

This is how Google Inbox works, about 70% of the code base is Java, which is the business logic, that runs on the Server, Web, Android, and iOS (via J2ObjC). The rest of the code is hand written JS, Java (Android), and Objective-C (iOS). For the Web, Closure reduces about 140Mb of unoptimized JS down to about 3mb uncompressed. The benefit we get from this system is a shared Java codebase across 4 platforms with pruning of the Java code that isn’t used.

1 Like