Feedback wanted -- Atrium an assertion library


#1

Hi everyone,
I just released a new version of my assertion library called Atrium:
https://github.com/robstoll/atrium
I think the project is now at a stage where I need some feedback.
Would be awesome if you could have a quick look and tell me what you think.
Do you miss a feature so that it stops you from using Atrium?
Any feedback is appreciated :slight_smile:


#2

Looks good.
First thing I saw is that there are no infix-functions.
Is this on purpose or maybe a future improvement?


#3

Good point, I’ll add it to the list of future improvements. Thanks :slight_smile:


#4

A follow-up question, how would you expect that an infix API looks like?

assert(1) toBe 2
assert(1) `to be` 2
1 toBe 2
1 `to be` 2

#5

I would prefer the toBe, but if you look at other assertion frameworks, I believe to be is the preferred choice


#6

and which one
assert(1) toBe 2
or
1 toBe 2


#7

I think with assert looks better.


#8

Thanks for all your feedback. You can get a sneak peek here from the infix API which will be available with v0.5.0:

Your feedback is again welcome :slight_smile:


#9

Just a note. I’ve just started to use Atrium along with Junit5 in a kotlin project.

I like it so far. I’ve added a few custom extensions for arrows Try.

Wondering if there are any plans to add some extensions for built in Arrow matchers, and perhaps something to do with JsonPath I could see being useful later.


#10

I don’t have particular plans to write assertion functions for Arrow as I don’t use the library myself. But please let me know what you need in particular and I’ll look into it. Ideally you share your existing functions :slight_smile:

Regarding JsonPath, I planned to add assertion functions for Json in 0.9.0:

Please have a look at it and provide feedback if you should miss a feature.


#11

Look forward to Json functions. I’ll let know if I have any actual thoughts :slight_smile:

In regard to Arrow, bear in mind that I am still relatively new to Kotlin, and understanding more each day.

What I’ve put together for Arrow Try matchers is:

import arrow.core.Try
import arrow.core.failure
import arrow.core.success
import ch.tutteli.atrium.creating.Assert
import ch.tutteli.atrium.creating.AssertionPlant
import ch.tutteli.atrium.domain.builders.AssertImpl
import ch.tutteli.atrium.reporting.RawString
import ch.tutteli.atrium.reporting.reporter
import ch.tutteli.atrium.reporting.translating.Untranslatable
import ch.tutteli.atrium.translations.DescriptionBasic
import ch.tutteli.atrium.verbs.assertThat

fun Assert<Try<Any>>.shouldBeSuccess() {
    createAndAddAssertion(
            DescriptionBasic.IS, RawString.create("a Success")) { subject.isSuccess() }
}

fun <A> Assert<Try<A>>.shouldBeSuccess(a: A) {
    val expected = a.success()
    createAndAddAssertion(
            DescriptionBasic.IS, expected) { subject == expected }
}

fun <A : Any> Assert<Try<A>>.shouldBeSuccess(assertionCreator: AssertionPlant<A>.() -> Unit) {
    AssertImpl.coreFactory.newReportingPlantAndAddAssertionsCreatedBy(
            Untranslatable("Success nested assertions"),
            { subject },
            reporter,
            {
                assertThat(subject).shouldBeSuccess()
                assertThat((subject as Try.Success).value, assertionCreator)
            }
    )
}

fun Assert<Try<Any>>.shouldBeFailure() {
    createAndAddAssertion(
            DescriptionBasic.IS, RawString.create("a Failure")) { subject.isFailure() }
}

fun Assert<Try<Any>>.shouldBeFailure(t: Throwable) {
    val expected = t.failure<Any>()
    createAndAddAssertion(
            DescriptionBasic.IS, expected) { subject == expected }
}

fun Assert<Try<Any>>.shouldBeFailure(assertionCreator: AssertionPlant<Throwable>.() -> Unit) {
    AssertImpl.coreFactory.newReportingPlantAndAddAssertionsCreatedBy(
            Untranslatable("Failure nested assertions"),
            { subject },
            reporter,
            {
                assertThat(subject).shouldBeFailure()
                assertThat((subject as Try.Failure).exception, assertionCreator)
            }
    )
}

inline fun <reified A : Throwable> Assert<Try<Any>>.shouldBeFailureOfType() {
    createAndAddAssertion(
            Untranslatable("is a Failure of type"), RawString.create(A::class.qualifiedName!!)) {
        when (subject) {
            is Try.Success -> false
            is Try.Failure -> (subject as Try.Failure).exception is A
        }
    }
}

Appreciate any feedback to, on how I’ve approached implementing the custom matchers :smile:


#12

I wouldn’t go with to be because it is more annoying to type and you get syntax completion later. The slightly better look isn’t worth it. Another argument against it is that you introduce more clutter to your implementation just to meke it look a bit more like natural language.

From my point of view DSLs are often totally overrated. Business users won’t read your assertions anyway. And for a programmer the difference doesn’t really matter since the rest of the test code doesn’t look like natural language anyway. There a so many libs out there that strive for english like code and have terrible, unitiuitive API with strange constructs to implement the syntax. Spend some time in Scala land and you will be cured :wink:


#13

Thanks for your thoughts, I went with toBe.


#14

Generally I recommend not to use assertThat within assertThat. You can, but if an inner assertThat fails it will throw an unexpected exception and the reporting is broken/does not behave like an assertion group. In your case you want to narrow the type from Try to Try.Success, for this you can use AssertImpl.any.typeTransformation.transform following an example for Either: https://github.com/robstoll/atrium/blob/master/domain/robstoll-lib/atrium-domain-robstoll-lib-jvm/src/test/kotlin/ch/tutteli/atrium/creating/any/typetransformation/creators/TypeTransformationAssertionCreatorSpec.kt#L41

The error reporting will be more stable and look a bit better. In the same sense I would build up shouldBeSuccess which expects an A by shouldBeSuccess which expects an assertionCreator as follows:

fun <A> Assert<Try<A>>.shouldBeSuccess(a: A)  = shouldBeSuccess { toBe(A) }

Same for shouldBeFailure.


#15

Thanks. Very useful tips. I’ve refactored my shouldBeSuccess and shouldBeFailure.

I know have something like:

fun <A : Any> Assert<Try<A>>.shouldBeSuccess(assertionCreator: AssertionPlant<A>.() -> Unit) {
    val parameterObject = AnyTypeTransformation.ParameterObject(
            Untranslatable("is a"),
            RawString.create(Success::class.java.simpleName),
            this,
            assertionCreator,
            Untranslatable("Could not evaluate the defined assertion(s) -- Try.isSuccess() was false")
    )
    AssertImpl.any.typeTransformation.transform(
            parameterObject, { it.isSuccess() }, { (it as Success).value },
            AssertImpl.any.typeTransformation.failureHandlers.newExplanatory()
    )
}

inline fun <reified A : Throwable> Assert<Try<Any>>.shouldBeFailure(noinline assertionCreator: AssertionPlant<A>.() -> Unit) {
    val parameterObjectFailure: AnyTypeTransformation.ParameterObject<Try<Any>, Throwable> = AnyTypeTransformation.ParameterObject(
            Untranslatable("is a"),
            RawString.create(Failure::class.java.simpleName),
            this,
            { },
            Untranslatable("Could not evaluate the defined assertion(s) -- Try.isFailure() was false")
    )
    AssertImpl.any.typeTransformation.transform(
            parameterObjectFailure, { it.isFailure() }, { (it as Failure).exception },
            AssertImpl.any.typeTransformation.failureHandlers.newExplanatory()
    )
    val parameterObjectThrowable: AnyTypeTransformation.ParameterObject<Try<Any>, A> = AnyTypeTransformation.ParameterObject(
            Untranslatable("is a"),
            RawString.create(A::class.java.simpleName),
            this,
            assertionCreator,
            Untranslatable("Could not evaluate the defined assertion(s) -- Failure was not of type")
    )
    AssertImpl.any.typeTransformation.transform(
            parameterObjectThrowable, { (it as Failure).exception is A }, { (it as Failure).exception as A },
            AssertImpl.any.typeTransformation.failureHandlers.newExplanatory()
    )
}

The Failure one, I am trying to allow assertions (for-now) on the actual expected error type. Not sure if there is a neater way.

But then, I have an example where I still find myself using a nested AssertThat. Not sure if there are better available methods to avoid this type of thing, or if I am not understanding the best way to use it yet.

assertThat(subject).shouldBeSuccess {
           assertThat(subject.details) {
                    property(subject::code).toBe("1")
                    property(subject::message).toBe("Message")
                    property(subject::detailMessage).notToBeNull {
                        assertThat(subject).containsStrictly("Invalid request")
                    }
            }
}

Thanks for taking the time to respond. I’m liking the library so far in my attempts to learn Kotlin :slight_smile:


#16

You can use isA<YourType>{} for regular subtyping assertions (see https://github.com/robstoll/atrium/blob/master/README.md#type-assertions). Assign the first call of AssertImpl.any.typeTransformation.transform to a variable, say plant. You can then define assertions for the new subject (of type Throwable in this case) via addAssertionsCreatedBy. Yet, there is a simpler way since ParamObject does exactly that. Currently you pass { } as assertionCreator. Which does nothing. Instead you could do:

{ 
  isA<A> { 
    addAssertionsCreatedBy (assertionCreator)
  }
}

For assertThat within assertThat. Use property as you already did or returnValueOf in case it is a method.

No need for assertThat for the following:

property(subject::detailMessage).notToBeNull {
                        assertThat(subject).containsStrictly("Invalid request")
                    }

You can use containsStrictly directly.


#17

That’s great. Thanks for the help.

Got some much nicer assertions now, that should help with my colleagues picking this up, who are even newer to Kotlin :slight_smile:


#18

No worries. Would be great if you could contribute back your assertion functions to Atrium - others using Arrow could certainly use it as well. I assume you are going to write more for other data types. A pull request on GitHub (or a private message with the code) would be more than welcome. I would then finalise your request so that it is available in all APIs and supports i18n. Looking forward to it :wink:


#19

No worries.

I will definitely try to get a PR created, when have a chance :slight_smile: