Unit testing in Kotlin/JS

Thanks! Asynchronous tests are not supported yet. Feel free to upvoute the feature request

Thanks Jørund! It should work for Maven as well. In order to compile the tests all you need is to have kotlin.test library as a dependency. Afterwards just use the compiled output normally with the testing framework of your choice.

Thank you for your feedback! Please upvote the feature request. Feel free to contribute any particular use-cases and pain points there as well.

As a temporal workaround for asynchronous tests you can do this:

https://github.com/korlibs/korio/commit/66c0799e11ef2907b2c765d326925fbc3b642074#diff-c197962302397baf3a4cc36463dce5eaR97

This reads the original .js file, search for test('...', false, function() { ... }) pattern and replaces it.
If the test function returns a js Promise, it returns it to the JS test framework, if not, it sets to null global.testPromise and then tries to read it before returning the test function.
That allows you to create unit tests that returns Unit/void compatible with junit.

And with an actual fun you can create something compatible with js unit test frameworks + junit.

https://github.com/korlibs/korio/commit/66c0799e11ef2907b2c765d326925fbc3b642074#diff-bfd9c14e9409d2e9178b873330d62eceR322

This doesn’t work well at all with multi-platform builds. As best I can tell, the process of copying modules into the kotlin2js destinationDir breaks other dependent libraries. Is there any support beyond this planned? It all seems very minimally designed and the different libraries and plugins (front-end plugin, etc) don’t work well together at all.

Thank you for the feedback. We are aware of some of the shortcomings, and do intend to fix them. So may I ask you to be a bit more specific, so that we don’t miss something?

What is the use case that’s broken?

Could you provide an example? Also please note that in the examples the modules are copied from kotlin2js destinationDir, not into. Also this is just one possible way to configure tests on Node, there is a number of other options. If you share your case, maybe I could suggest a solution.

Could you expand on this as well?

Sorry for asking so many questions, but unless I understand, how you are using it and what is causing the pain, we won’t be able to take it into account when designing the fix.

@steamstreet Hi! Looking back my questions might have sounded a bit aggressive. Sorry for that =)

I am genuinely interested in the specific things that don’t work well at this moment. I know about some of them - complicated setup (especially multiplatform async tests), non-uniform IDE support, etc. Problem is, most of the feedback comes from our library team, in particular kotlinx.coroutines. That means we might be not aware of some particular problems they have not encountered.

We are planning to have an internal discussion about those in about a week or two. Would you like to share your use cases and pain points, so that we make sure to take them into account?

Also if you miss some features or experience of some other language/framework/etc., you are welcome to share that as well =)

I’ll share something I was running into yesterday - I can write async tests for the jvm and native by using the runblocking function from the coroutines library. However, the only way I’ve gotten async tests to work in javascript was by taking advantage of the underlying library’s “return a promise” functionality. Which works great!.. except the tests written in kotlin have to have a non-Unit return type.

Which means that the jvm and native test runners barf when encountering these tests (even if in practice nothing is returned on those platforms thanks to external/actual function work).

So that’s a hard blocker for writing multiplatform async tests between the js and everything else.

Basically, I’d want the ability to write an async test in javascript that doesn’t have to rely on returning a value, as that seems like something the generic test framework didn’t anticipate.

I just ran my first Kotlin/JS unit test and I want to thank you @Anton.Bannykh for providing the links and sample project as they were indispensable.

However I have to say that this experience was miserable. I tried qunit, mocha and tape but all failed with “window” undefined. So I did some digging and found jsdom and global-jsdom which can be made to provide a headless virtual browser to mocha. However, jsdom does not support the Fetch API (window.fetch( …)). Unfortunately these frameworks all run your complete JS through them so even if you just want to assert 1==1 that will fail if you have fetch anywhere in your code.

In the end I was able to make Karma work with gradle test. The trick was to use a later Node version because Karma would fail with async start() Symbol Expected error.

Add this to gradle:

node {
// Version of node to use.
version = ‘8.9.4’
download = true
// Set the work directory for unpacking no>de
workDir = file("{project.buildDir}/nodejs")     nodeModulesDir = file("{project.projectDir}")
}

Of course (because OF COURSE) it only works with 8.9.4 because later node version have a different distribution layout which the gradle node plugin doesn’t like.

The Karma sample project uses browser “PhantomJs” which also doesn’t work for me because … it doesn’t support the fetch API. So ‘chrome’ it is, at least that works!

Here is some flamebait: The JavaScript ecosystem is a dumpster fire and I wish kotlin/js would help me to stay away from it. I just wanted to execute some tests and had to spent 2 days chasing outdated framework versions, switched library names ( there is a global-jsdom and a jsdom-global lib …) and incompatible node versions. Please, Jetbrains, please just make it work seemlessly through Intellij.

@wulf0r Sorry to hear about your experience.

The good news is that @Sergey.Rostov is working exactly improving the testing experience: https://youtrack.jetbrains.com/issue/KT-30526 (that’s an umbrella issue)

As far as I know configuring NodeJS manually should become deprecated starting from 1.3.40: https://youtrack.jetbrains.com/issue/KT-30528

Not sure if we are going to support headless chrome out of the box. (cc @Sergey.Rostov)

2 Likes

Hi! Yes, as @Anton.Bannykh just said, we are currently working on test running. Running tests on browser with Karma currently planned for 1.3.40, but I cannot make any promises (https://youtrack.jetbrains.com/issue/KT-31011). I hope this will work out of the box when it is released.

2 Likes

@Anton.Bannykh @Sergey.Rostov Thank! Keep up the great work and thanks for everything!

@Anton.Bannykh @Sergey.Rostov Is it possible to invoke the suite and test functions directly, rather than using the annotations? I ask because if so it would be possible to use KotlinTest as a JS test framework.

@sksamuel I think it’s possible. As far as I know the kotlin-test library only depends on the kotlin stdlib. Also it shouldn’t require any configuration out of the box.

The suite and test are located inside kotlin.test package, i.e. require('kotlin-test').kotlin.test.suite returns the suite function.

It seems the compiler has special support for @Test and @Ignore, and generates top level calls for these inside the generated javascript.

Eg,

  package$js.main = main;
  package$js.FibTestsKotlinTest = FibTestsKotlinTest;
  package$js.FibTestsSdk = FibTestsSdk;
  package$js.FunSpec = FunSpec;
  suite('io.kotlintest.example.mpp.js', false, function () {
    suite('FibTestsSdk', false, function () {
      test('fibTests', false, function () {
        return (new FibTestsSdk()).fibTests();
      });
    });
  });
  main();
  Kotlin.defineModule('kotlintest-example-multiplatform_test', _);
  return _;

If I could find out a way to also generate code at the top level like this, then I think I would have enough to implement KotlinTest for JS.

@sksamuel The important bit is suite(...) The suite and test invocations don’t have to be top-level. They just need to be invoked at some point after module initialization.

I don’t think there is a sane way to achieve that though. The following text describes an approach which might work, but requires quite some effort to implement.

One possibility is to abuse test inheritance. Basically it is possible to inherit tests from a base class or interface (see this test)

So I think you could try doing the following:

  • Add a @Test dummyTest() { <launch your custom tests here> } to StringSpec
  • The compiler will generate a test("dummyTest", false, ...) for each successor
  • When dummyTest is invoked launch your custom tests

There are at least two problems with this scheme

  • You get some garbage in the test hierarchy (“dummyTest”)
  • I suspect that test frameworks might not like suites within tests

In order to solve those issues you might want to replace the test adapter with some custom implementation, which would filter out the dummyTest tests and delegate to the original test adapter when needed. (example of complex test adapter)

Your suggestion is exactly what I did last night. Custom framework adapter that looks for the dummy name and instead calls other tests.

Good job )

Do you know if there’s a way to add the adapter.js file from a dependency, so it doesn’t have to be added manually to every proejct ?

Why not just set up the adapter during KotlinTest initialization? Am I missing something?

I mean, whoever is going to use KotlinTest will require it. Thus changing the test adapter during KotlinTest module initialization seems like a perfectly good solution to me.

Also the test adapter doesn’t have to a separate .js file. It can be done in Kotlin, albeit you’ll have to jump through some hoops.

If you do have it in a separate .js file, why not make the KotlinTest module depend on in? I.e. you require the adapter.js, it configures kotlin-test.