Best test framework for Kotlin

There are at least three test frameworks with specific Kotlin support, and I wonder which one to choose:

Spek - “A Kotlin Specification Framework for the JVM”

  • Though Spek is not an official JetBrains project, it is hosted on the JetBrains GitHub account and has contributors from JetBrains.
  • 814 GitHub stars

KotlinTest - “Kotlin test framework based on the excellent Scalatest”

  • 471 GitHub stars

JUnit + HamKrest - “Hamcrest for Kotlin”

  • While there is no Kotlin-specific version of JUnit itself, there is HamKrest.
  • It appears to me that the Kotlin standard library is tested with JUnit.
  • 142 GitHub stars (HamKrest)

All these options and also TestNG are approved of in the book Kotlin in Action.

So I wonder which of these frameworks I should pick for a new Kotlin project - Spek, KotlinTest or JUnit5 + HamKrest? I have experience with JUnit4 + Hamcrest in Java and I know Spek offers “BDD-style” tests. But I have no idea which of these frameworks is best/state-of-the-art, the most future-proof or the most “idiomatic” for Kotlin. Maybe the ideal framework even depends on the kind of test, e.g. unit test vs. end-to-end test?

1 Like

My personal preference is Spek + AssertJ or more classical TestNG + AssertJ.

I would look at some more spicy things like parametrized tests (table-driven tests, property based tests), interceptors or grouping tests by tags and see what framework fits your needs. However, I think KotlinTest will do a really good job here.

2 Likes

I tried MockK library, it’s awesome, verify similar to Mockito/Powermock and IT WORKS!
In additional, the Genius (the library owner - Oleksiy Pylypenko) of this library is available for chat most of the time on gitter.im and helps me a lot !
Strongly recommended if you need assert,mock,staticMock, verify, test coroutines code and much more

https://github.com/oleksiyp/mockk

2 Likes

I missed test parameterisation and creating reusable testing suites. There is some support from JUnit5, but it feels kinda clunky with annotations and stuff; I also never felt at home with Spek assigning semantics to tests. And hence I use DynaTest to structure my tests as I see fit; then I use Kotlin Test for asserts.

It is fine if you’re happy with DynaTest, but I cannot see how this framework is more powerful than KotlinTest.

DynaTest allows for arbitrary grouping of tests, with unlimited group depth. That combined with beforeGroup and afterGroup blocks creates possibilities to group tests and setup environment for the tests properly. You can also create groups of tests dynamically, with a parametrized function call, hence parametrized tests. I believe kotlin test lacks groups if I’m not mistaken.

What do you mean by group exactly? Just the ability to arbitrarily nest tests? If so, then FreeSpec will do that.

https://github.com/kotlintest/kotlintest/blob/master/kotlintest-tests/kotlintest-tests-core/src/test/kotlin/com/sksamuel/kotlintest/specs/freespec/FreeSpecExample.kt

Then you can use a listener to be notified on each level:
https://github.com/kotlintest/kotlintest/blob/master/kotlintest-core/src/main/kotlin/io/kotlintest/extensions/TestListener.kt

You can even control if a test (group) is executed at all:
https://github.com/kotlintest/kotlintest/blob/master/kotlintest-core/src/main/kotlin/io/kotlintest/extensions/TestCaseExtension.kt