Calling Kotlin's compiler from another plugin?

Hi,

I’m trying to call the Kotlin compiler from another plugin (in Android Studio) to compile a selection of kt files. I tried a couple of things with no luck yet:

  1. directly using “org.jetbrains.jet.cli.jvm.K2JVMCompiler” will result in a NoClassDefFoundError.
  2. when I create a custom class loader and load the above class myself (and use reflection to call “exec()”), the IDE just freezes with a whole bunch of seeming unrelated exceptions such as below.

What class should I use instead? I’d appreciate any pointer or suggestions!

java.lang.ClassCastException: com.intellij.mock.MockApplicationEx cannot be cast to com.intellij.openapi.application.impl.ApplicationImpl
at com.intellij.idea.IdeaLogger.logErrorHeader(IdeaLogger.java:161)
at com.intellij.idea.IdeaLogger.error(IdeaLogger.java:142)
at com.intellij.openapi.diagnostic.Logger.error(Logger.java:132)
at com.intellij.ide.plugins.PluginManager.processException(PluginManager.java:120)
at com.intellij.ide.IdeEventQueue.defaultDispatchEvent(IdeEventQueue.java:701)
at com.intellij.ide.IdeEventQueue._dispatchEvent(IdeEventQueue.java:524)
at com.intellij.ide.IdeEventQueue.dispatchEvent(IdeEventQueue.java:335)
at java.awt.EventDispatchThread.pumpOneEventForFilters(EventDispatchThread.java:296)
at java.awt.EventDispatchThread.pumpEventsForFilter(EventDispatchThread.java:211)
at java.awt.EventDispatchThread.pumpEventsForHierarchy(EventDispatchThread.java:201)
at java.awt.EventDispatchThread.pumpEvents(EventDispatchThread.java:196)
at java.awt.EventDispatchThread.pumpEvents(EventDispatchThread.java:188)
at java.awt.EventDispatchThread.run(EventDispatchThread.java:122)

and

java.lang.NullPointerException
at com.intellij.openapi.editor.impl.SettingsImpl.isRightMarginShown(SettingsImpl.java:88)
at com.intellij.openapi.editor.impl.EditorImpl.paintRightMargin(EditorImpl.java:2123)
at com.intellij.openapi.editor.impl.EditorImpl.paint(EditorImpl.java:1981)
at com.intellij.openapi.editor.impl.EditorComponentImpl.paintComponent(EditorComponentImpl.java:153)
at javax.swing.JComponent.paint(JComponent.java:1037)
at javax.swing.JComponent._paintImmediately(JComponent.java:5106)
at javax.swing.JComponent.paintImmediately(JComponent.java:4890)
at javax.swing.RepaintManager$3.run(RepaintManager.java:814)
at javax.swing.RepaintManager$3.run(RepaintManager.java:802)
at java.security.AccessController.doPrivileged(Native Method)
at java.security.AccessControlContext$1.doIntersectionPrivilege(AccessControlContext.java:86)
at javax.swing.RepaintManager.paintDirtyRegions(RepaintManager.java:802)
at javax.swing.RepaintManager.paintDirtyRegions(RepaintManager.java:745)
at javax.swing.RepaintManager.prePaintDirtyRegions(RepaintManager.java:725)
at javax.swing.RepaintManager.access$1000(RepaintManager.java:46)

First of all, about the root of your problems:

Kotlin IDE plugin does not call the compiler within the same process, so you shouldn’t do that either. The compiler has a mick implementation of the core components of IntelliJ IDEA, and when you load them into the process where normal implementations are already loaded, they fight desperately, which results in what you are getting.

What you can do:

  • Call the compiler as an external process. This is what Kotlin IDE integratio does itself (same as what;s done for Java nad otehr languages in IntelliJ IDEA)
  • Call the compiler in the process of IntelliJ IDEA, but using an isolated class loader

Thanks for the quick response Andrey.

I’m trying the second method but got another NoClassDefFoundError on org/picocontainer/MutablePicoContainer. I understand though starting it from a separate process should just work.

I created a class loader with its parent being kotlinPlugin.getPluginClassLoader().getSystemClassLoader(), and “kotlin-compiler.jar”, “kotlin-runtime.jar” and “kotlin-preloader.jar” are all on classpath. Below is the full stacktrace.

It’s different than what’s in kotlinc-jvm but I suppose my method should work too? Where is this MutablePicoContainer located?

java.lang.NoClassDefFoundError: org/picocontainer/MutablePicoContainer
at java.lang.ClassLoader.defineClass1(Native Method)
at java.lang.ClassLoader.defineClassCond(ClassLoader.java:637)
at java.lang.ClassLoader.defineClass(ClassLoader.java:621)
at java.security.SecureClassLoader.defineClass(SecureClassLoader.java:141)
at java.net.URLClassLoader.defineClass(URLClassLoader.java:283)
at java.net.URLClassLoader.access$000(URLClassLoader.java:58)
at java.net.URLClassLoader$1.run(URLClassLoader.java:197)
at java.security.AccessController.doPrivileged(Native Method)
at java.net.URLClassLoader.findClass(URLClassLoader.java:190)
at java.lang.ClassLoader.loadClass(ClassLoader.java:306)
at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:301)
at java.lang.ClassLoader.loadClass(ClassLoader.java:247)
at java.lang.ClassLoader.defineClass1(Native Method)
at java.lang.ClassLoader.defineClassCond(ClassLoader.java:637)
at java.lang.ClassLoader.defineClass(ClassLoader.java:621)
at java.security.SecureClassLoader.defineClass(SecureClassLoader.java:141)
at java.net.URLClassLoader.defineClass(URLClassLoader.java:283)
at java.net.URLClassLoader.access$000(URLClassLoader.java:58)
at java.net.URLClassLoader$1.run(URLClassLoader.java:197)
at java.security.AccessController.doPrivileged(Native Method)
at java.net.URLClassLoader.findClass(URLClassLoader.java:190)
at java.lang.ClassLoader.loadClass(ClassLoader.java:306)
at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:301)
at java.lang.ClassLoader.loadClass(ClassLoader.java:247)
at com.intellij.openapi.extensions.impl.ExtensionsAreaImpl.<init>(ExtensionsAreaImpl.java:67)
at com.intellij.openapi.extensions.Extensions.createRootArea(Extensions.java:42)
at com.intellij.openapi.extensions.Extensions.<clinit>(Extensions.java:38)
at com.intellij.core.CoreApplicationEnvironment.<init>(CoreApplicationEnvironment.java:84)
at com.intellij.core.JavaCoreApplicationEnvironment.<init>(JavaCoreApplicationEnvironment.java:54)
at org.jetbrains.jet.cli.jvm.compiler.JetCoreEnvironment.createApplicationEnvironment(JetCoreEnvironment.java:155)
at org.jetbrains.jet.cli.jvm.compiler.JetCoreEnvironment.getOrCreateApplicationEnvironmentForProduction(JetCoreEnvironment.java:127)
at org.jetbrains.jet.cli.jvm.compiler.JetCoreEnvironment.createForProduction(JetCoreEnvironment.java:101)
at org.jetbrains.jet.cli.jvm.K2JVMCompiler.doExecute(K2JVMCompiler.java:157)
at org.jetbrains.jet.cli.jvm.K2JVMCompiler.doExecute(K2JVMCompiler.java:48)
at org.jetbrains.jet.cli.common.CLICompiler.exec(CLICompiler.java:142)
at org.jetbrains.jet.cli.common.CLICompiler.exec(CLICompiler.java:122)
at org.jetbrains.jet.cli.common.CLICompiler.exec(CLICompiler.java:52)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at com.jimulabs.mirror.plugin.IdeaKotlinCompiler.compile(IdeaKotlinCompiler.java:49)
at com.jimulabs.mirror.server.SandboxCompiler.compileKotlin(SandboxCompiler.java:170)
at com.jimulabs.mirror.server.SandboxCompiler.compileSrc(SandboxCompiler.java:123)
at com.jimulabs.mirror.server.SandboxCompiler.compileInBatches(SandboxCompiler.java:108)
at com.jimulabs.mirror.server.SandboxCompiler.compile(SandboxCompiler.java:72)
at com.jimulabs.mirror.server.MirrorServer.prepareSandboxDex(MirrorServer.java:119)
at com.jimulabs.mirror.server.MirrorServer.access$300(MirrorServer.java:24)
at com.jimulabs.mirror.server.MirrorServer$4.onChange(MirrorServer.java:104)
at com.jimulabs.mirror.server.AndroidProjectMonitor$FileChangeListener.onStop(AndroidProjectMonitor.java:298)
at org.apache.commons.io.monitor.FileAlterationObserver.checkAndNotify(FileAlterationObserver.java:313)
at org.apache.commons.io.monitor.FileAlterationMonitor.run(FileAlterationMonitor.java:182)
at java.lang.Thread.run(Thread.java:695)
Caused by: java.lang.ClassNotFoundException: org.picocontainer.MutablePicoContainer
at java.net.URLClassLoader$1.run(URLClassLoader.java:202)
at java.security.AccessController.doPrivileged(Native Method)
at java.net.URLClassLoader.findClass(URLClassLoader.java:190)
at java.lang.ClassLoader.loadClass(ClassLoader.java:306)
at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:301)
at java.lang.ClassLoader.loadClass(ClassLoader.java:247)
… 53 more

I think you should not specify any parent for your class loader, but to diagnose further you'll need to use your debugger

worked like a charm after dropping the parent classloader! Thanks Andrey!

BTW, what are the advantages of using a separate process? Isn't it slower than running the compiler in process?

This is a generic decision taken in IntelliJ IDEA, so I might be not fully informed on the motivation, but I see the following reasons:

  • memory: a compilation process might not fit into IDE's heap
  • compatibility: the build system used internally by the IDE is also used when building IntelliJ IDEA projects on TeamCity build server, where it's run as a process

Thanks for the great answer, Andrey!

BTW: Just to compelete the question so it’d be useful for others, I switched to use a separate process as well since I got another set of strange errors when trying to invoke the compiler in the same process.