Unit testing in Kotlin/JS

A recent 1.1.4-eap-33 release provides improvements to the core library kotlin.test and the compiler. The changes include out-of-the-box support for Jasmine, Mocha, and Jest, and an experimental way to plug in a custom unit testing framework.

Usage

A simple test might look like so:

package foo
import kotlin.test.*

class Test {
  @Test fun bar() {
    assertEquals(4, 2 * 2)
  }

  @Ignore fun baz() {
    fail("Doesn't work")
  }
}

Mark the test functions with the kotlin.test.Test annotation. To run the tests pass a compiled .js file to a unit testing framework. No configuration is required for supported frameworks. The kotlin-test-js library detects the framework at runtime and uses the corresponding API. The details of using other unit testing framework will be documented later.

The other new annotation is kotlin.test.Ignore. Classes and test functions marked with this annotation are treated as ‘pending’ by the unit testing frameworks (meaning that they don’t execute).

Please check out the sample Gradle projects for QUnit, Jasmine, Mocha, Jest, Karma, frontend-plugin, and Tape. The last one shows how to add support for a custom unit testing framework. To run the tests in each of the sample projects, execute the following command:

./gradlew test

There is also a sample project for IntelliJ IDEA. To run the tests in the IDE you need the Ultimate version with NodeJs plugin installed.

Breaking changes

If you are already using the kotlin-test-js library with QUnit the changes should not affect you except for some rare cases:

  • The QUnitAsserter class has been removed, so the library update will affect your setup if you are using it.
  • Annotation org.junit.Test is deprecated, but still supported. It is a type alias to kotlin.test.Test now. Its parameter name was removed.
2 Likes

Looking at samples and kjsm files in kotlin-test-js it seem like there is no async test support in api. Please consider adding such api (via callbacks and/or promises), it is essential for js testing.

Your Tape link points to frontend-plugin instead of https://github.com/JetBrains/kotlin-examples/tree/master/gradle/js-tests/tape

Fixed. Thanks!

Nice work, guys! Just a question. Did this test library support asynchronous tests (when we return a promise)?

Hi!

Thanks for moving forward with improving support of unit-testing of Kotlin/JS code.

I’ve been looking through the examples, and ufortunately I cannot find any examples using Maven. Does these improvements only affects Gradle projects?

Regards,
Jørund

Thanks! Asynchronous tests are not supported yet. Feel free to upvoute the feature request

Thanks Jørund! It should work for Maven as well. In order to compile the tests all you need is to have kotlin.test library as a dependency. Afterwards just use the compiled output normally with the testing framework of your choice.

Thank you for your feedback! Please upvote the feature request. Feel free to contribute any particular use-cases and pain points there as well.

As a temporal workaround for asynchronous tests you can do this:

https://github.com/korlibs/korio/commit/66c0799e11ef2907b2c765d326925fbc3b642074#diff-c197962302397baf3a4cc36463dce5eaR97

This reads the original .js file, search for test('...', false, function() { ... }) pattern and replaces it.
If the test function returns a js Promise, it returns it to the JS test framework, if not, it sets to null global.testPromise and then tries to read it before returning the test function.
That allows you to create unit tests that returns Unit/void compatible with junit.

And with an actual fun you can create something compatible with js unit test frameworks + junit.

https://github.com/korlibs/korio/commit/66c0799e11ef2907b2c765d326925fbc3b642074#diff-bfd9c14e9409d2e9178b873330d62eceR322

This doesn’t work well at all with multi-platform builds. As best I can tell, the process of copying modules into the kotlin2js destinationDir breaks other dependent libraries. Is there any support beyond this planned? It all seems very minimally designed and the different libraries and plugins (front-end plugin, etc) don’t work well together at all.

Thank you for the feedback. We are aware of some of the shortcomings, and do intend to fix them. So may I ask you to be a bit more specific, so that we don’t miss something?

What is the use case that’s broken?

Could you provide an example? Also please note that in the examples the modules are copied from kotlin2js destinationDir, not into. Also this is just one possible way to configure tests on Node, there is a number of other options. If you share your case, maybe I could suggest a solution.

Could you expand on this as well?

Sorry for asking so many questions, but unless I understand, how you are using it and what is causing the pain, we won’t be able to take it into account when designing the fix.

@steamstreet Hi! Looking back my questions might have sounded a bit aggressive. Sorry for that =)

I am genuinely interested in the specific things that don’t work well at this moment. I know about some of them - complicated setup (especially multiplatform async tests), non-uniform IDE support, etc. Problem is, most of the feedback comes from our library team, in particular kotlinx.coroutines. That means we might be not aware of some particular problems they have not encountered.

We are planning to have an internal discussion about those in about a week or two. Would you like to share your use cases and pain points, so that we make sure to take them into account?

Also if you miss some features or experience of some other language/framework/etc., you are welcome to share that as well =)

I’ll share something I was running into yesterday - I can write async tests for the jvm and native by using the runblocking function from the coroutines library. However, the only way I’ve gotten async tests to work in javascript was by taking advantage of the underlying library’s “return a promise” functionality. Which works great!.. except the tests written in kotlin have to have a non-Unit return type.

Which means that the jvm and native test runners barf when encountering these tests (even if in practice nothing is returned on those platforms thanks to external/actual function work).

So that’s a hard blocker for writing multiplatform async tests between the js and everything else.

Basically, I’d want the ability to write an async test in javascript that doesn’t have to rely on returning a value, as that seems like something the generic test framework didn’t anticipate.

I just ran my first Kotlin/JS unit test and I want to thank you @Anton.Bannykh for providing the links and sample project as they were indispensable.

However I have to say that this experience was miserable. I tried qunit, mocha and tape but all failed with “window” undefined. So I did some digging and found jsdom and global-jsdom which can be made to provide a headless virtual browser to mocha. However, jsdom does not support the Fetch API (window.fetch( …)). Unfortunately these frameworks all run your complete JS through them so even if you just want to assert 1==1 that will fail if you have fetch anywhere in your code.

In the end I was able to make Karma work with gradle test. The trick was to use a later Node version because Karma would fail with async start() Symbol Expected error.

Add this to gradle:

node {
// Version of node to use.
version = ‘8.9.4’
download = true
// Set the work directory for unpacking no>de
workDir = file(“${project.buildDir}/nodejs”)
nodeModulesDir = file(“${project.projectDir}”)
}

Of course (because OF COURSE) it only works with 8.9.4 because later node version have a different distribution layout which the gradle node plugin doesn’t like.

The Karma sample project uses browser “PhantomJs” which also doesn’t work for me because … it doesn’t support the fetch API. So ‘chrome’ it is, at least that works!

Here is some flamebait: The JavaScript ecosystem is a dumpster fire and I wish kotlin/js would help me to stay away from it. I just wanted to execute some tests and had to spent 2 days chasing outdated framework versions, switched library names ( there is a global-jsdom and a jsdom-global lib …) and incompatible node versions. Please, Jetbrains, please just make it work seemlessly through Intellij.

@wulf0r Sorry to hear about your experience.

The good news is that @Sergey.Rostov is working exactly improving the testing experience: https://youtrack.jetbrains.com/issue/KT-30526 (that’s an umbrella issue)

As far as I know configuring NodeJS manually should become deprecated starting from 1.3.40: https://youtrack.jetbrains.com/issue/KT-30528

Not sure if we are going to support headless chrome out of the box. (cc @Sergey.Rostov)

2 Likes

Hi! Yes, as @Anton.Bannykh just said, we are currently working on test running. Running tests on browser with Karma currently planned for 1.3.40, but I cannot make any promises (https://youtrack.jetbrains.com/issue/KT-31011). I hope this will work out of the box when it is released.

2 Likes

@Anton.Bannykh @Sergey.Rostov Thank! Keep up the great work and thanks for everything!

@Anton.Bannykh @Sergey.Rostov Is it possible to invoke the suite and test functions directly, rather than using the annotations? I ask because if so it would be possible to use KotlinTest as a JS test framework.

@sksamuel I think it’s possible. As far as I know the kotlin-test library only depends on the kotlin stdlib. Also it shouldn’t require any configuration out of the box.

The suite and test are located inside kotlin.test package, i.e. require('kotlin-test').kotlin.test.suite returns the suite function.