KSpec - Specifications for Kotlin

Libraries
1

Hey guys, I started a specification framework for kotin - GitHub - raniejade/kspec: Kotlin Specification Framework. It’s nothing big I just need something to practice on to improve my knowledge about Kotlin. For samples, you can check the tests.

DSL is heavily inspired by GitHub - Quick/Quick: The Swift (and Objective-C) testing framework..

class EqualsSpec: KSpec() {
    override fun spec() {
        describe("Equals") {
            val matcher = Equals(1, null)

            describe("match") {
                context("passed argument is not equal to expected") {
                    it("should throw an AssertionError") {
                        expect({
                            matcher.match(2)
                        }).toBe(thrown(AssertionError::class))
                    }
                }

                context("passed argument is equal to expected") {
                    it("should not throw an exception") {
                        expect({
                            matcher.match(1)
                        }).toBe(notThrown())
                    }
                }
            }
        }
    }
}
2

Just to make sure you are aware that JetBrains has Spek (GitHub - spekframework/spek: A specification framework for Kotlin) but personally I am not impressed by it. You seem to have gone much farther than Spek.

A couple comments from quick perusal:

  • What’s up with truthy and falsy methods? Are you trying to be funny? Give them real names.
  • You have created your own set of matchers, but you should use the standard libraries for this, Hamcrest or the Kotlin version HamKrest
3

I’d prefer AssertJ. It is more readable than Hamcrest style matchers and produces better error messages.

4

Which is ridiculous, because Hamcrest has very good descriptions for its matchers that can even be queried for their description without running the matcher. The matchers in this library only describe themselvees on failure.

The real issue here is that the abilities created here are primitive compared to the wide variety of powerful Matchers available in Hamcrest.

I am not even saying that you have to eliminate what is here, just allow Hamcrest matchers to be used as well. Although with Kotlin extension functions, ti is probably trivial to add it myself.

5

Matchers are purely experimental and optional, you are not required to use them. You can use your preferred library using extension functions (https://github.com/raniejade/kspec/blob/master/src/main/kotlin/kspec/asserts/Asserts.kt).

6

Was going to come back and suggest that you at least move the matcher stuff into a separate lib, but now see that you have removed it.

Now without the matcher stuff, there actually IMO is no need for the It interface. The only method defined on It is fail, which isn’t really needed. You can just use Assert.fail from Junit. If you really felt it was necessary you could move it to the Spec interface.

With It gone then the it function in spec just takes a () → Unit instead of an It.() → Unit and there really is no significance between describe, context, and it other than it is a terminal. describe, context, and it are basically just text strings put into the junit test descriptions.

So it seems to me that Context.terminal should not be a value you set on the context and instead is just:

val terminal : Boolean get() = children.isEmpty

So that any context without children is terminal.

You should then allow people to use any words they want instead of just describe, context, and it. For example, people will want the ability to use the given, when, then verbiage.

To accomplish this Spec, should have a method of this form (not sure what the best name is):

fun specBlock(term: String? = null, description: String, action: () -> Unit)

I would then move describe, context, and it from the interface to instead be inline extension functions that just call this method. If someone wants to use given-when-then they can define extension functions with these names that similarly call this method.

Note, I made term nullable so that if it is null you do not add the term, colon, and space in front of the junit description so that you get just the description.

7

Thanks for the feedback, I can get some ideas from here.

Terminality must be known before the context (block/clause) is evalutated. Basically the JUnit runner has two phases: collection and execution. The former collects/discovers tests (terminal context) and the latter executes the tests, including all the fixtures (before, after, beforeEach and afterEach) of the context hierarchy. The side-effect of the collection phase is that the non-terminal contexts are evaluated. With that said the method would look like:

fun specBlock(description: String, term: String? = null, terminal: Boolean = false, block: () -> Unit)

The current terms can be implemented as extension functions.

fun Spec.describe(description: String, action: () -> Unit) {
    specBlock(description, "describe", false, action)
}

fun Spec.context(description: String, action: () -> Unit) {
    specBlock(description, "context", false, action)
}

fun Spec.it(description: String, action: () -> Unit) {
    specBlock(description, "it", true, action)
}
8

IF terminal parameter were needed would make more sense to split it into terminalBlock and nonTerminalBlock rather than use a paramter.

However, I don’t think it is needed. I ran a test of changing the start of Context to this:

class Context(var description: String, var action: (Context) -> Unit,
          val parent: Context?, val terminal_: Boolean = false) {

val terminal : Boolean
    get(){
        assert(terminal_ == children.isEmpty())
        return terminal_
    }

And it never failed

9

Ahhh… Indeed the context class can check the children property for terminality. However we still need the terminal parameter in specBlock, see https://github.com/raniejade/kspec/blob/master/src/main/kotlin/io/polymorphicpanda/kspec/KSpecEngine.kt#L16. In the collection phase we don’t want to evaluate the terminal contexts but just collect them.

10

OK, I have read more about the differences between examples and example groups in relation to RSpec and now understand the significance of what terminal was supposed to mean and would like to propose these changes:

Eliminate Context.terminal and replace it with whether the Context has children

Get rid of the terminal parameter and instead split specBlock into 2 separate methods following the RSpec terminology, example for cases where terminal was true and group (or possibly exampleGroup) for cases where terminal was false.

Get rid of the term parameter and just let the extension functions take care of the formatting as in:

inline fun Spec.describe(description: String, action: () -> Unit)
    = group("describe: $description", action)

Note that these should be inline.

Also now that It.fail is gone, Context.failure is not needed anymore

11

Let me read through RSpec first, it might take a while though. I’ll be on mobile (away from my pc) for a week.

12

So, I forked your library and made the changes I suggested, and went hog wild on refactoring it and added some additional cool features. It is now a very lean and Kotlin-esque library.

You can see my changes here (sorry I refuse to put any code on GitHub so I cannot do a pull request): Commits · refactoring · Dale King / KSpec · GitLab

Unfortunately I did not try to maintain the unit test through the process. You can incorporate the changes in your codebase or I will maintain my fork.

13

I repushed the branch with fixes for the unit tests that were there

14

Just got back, this looks very interesting. Let me check it out, thanks!

15

Don’t look at it too closely yet as I am in the process of redoing a lot of it. Some of the things I did like remove the visitor pattern and JUnitTestDescriber and JUnitTestExecutor I am actually going to undo. And I realize after more research that the way I did the subject stuff is wrong, it should not appear in test hierarchy and right now it is more like a before block when it really needs to be more of a beforeEach, which complicates the execution logic immensely and I am having trouble even wrapping my brain around how it will work.

16

I’ll also try to experiment/implement the concepts you’ve introduced using the current master as the base.

17

So I decided to release version 0.1.0 which is available at jcenter. You can check the available features at GitHub - raniejade/kspec: Kotlin Specification Framework. Feel free to test it out and as always feedback is much appreciated.

18

I don’t think your implementation of subjects is correct. I haven’t forgotten about KSpec, but have been too busy to work on it lately