Strange issues when running multiple JUnit tests in Intellij IDEA - 1.0.4+

Support
1

Hi,

Since 1.0.4 (and on 1.0.5 EAP, which I’m running currently), I’m getting strange errors when running JUnit tests in IDEA:

java.lang.NoSuchMethodError: org.hamcrest.Matcher.describeMismatch(Ljava/lang/Object;Lorg/hamcrest/Description;)V

	at org.hamcrest.MatcherAssert.assertThat(MatcherAssert.java:18)
	at org.hamcrest.MatcherAssert.assertThat(MatcherAssert.java:8)
	at myProject.myTest(MyTests.kt:83)
	... more stuff

The stack trace points to a ‘NoSuchMethodError’ in Hamcrest, a fairly stable & commonly used library. This issues occurs ‘randomly’ in the sense that I get the same error multiple times, but inconsistently - sometimes a test will pass then fail with this error on the next run. If I run the tests in that class in isolation (i.e. not when running all my other tests), then it passes consistently The issue seems to only occur when I run all my tests.

Similarly, I have a number of tests which use JUnit Rules and ClassRules. I didn’t have any problems in Kotlin 1.0.3 (and/or the version of IDEA which was around at the time, I think 2016.2.2), but now when I run all my tests, it’s as if the Rules were ignored - the ExpectedExceptions are not handled, or the data set up is not performed. Again, when I run the tests individually, things work as expected.

If I run the tests in Gradle (complied with Kotlin 1.0.3), things work as expected, even when running the whole suite (although I have a multi-project build, so Gradle runs the tests in each project separately, unlike IDEA).

Any ideas as to what could be causing this?

Thanks,
Nick

2

When IDEA runs your tests, it outputs a full command line, including a classpath, to the output toolwindow. You can check that the command line references the correct version of Hamcrest.

Also, is your project doing something non-standard with classloaders?

3

Thanks very much for getting back - the issue was with a dependency on Google’s closure templates I’d added around the same time as I’d upgraded Kotlin. That JAR file had a dependency on junit 4.8.2, which in turn includes an outdated version of the Hamcrest matchers. The dependency was in a shared project, which must be why it fails when running tests in all projects (including the shared project, which therefore causes the incorrect version of JUnit to be included) but not when the tests are run individually.

The issue is therefore entirely unrelated to Kotlin. The fix was to exclude the JUnit dependency in gradle:

compile (group: 'com.google.template', name: 'soy', version: '2016-08-25') {
    exclude group: 'junit'
}

Thanks again for putting me on the right track -
Nick