fyi Kotlin implementations of benchmarks game programs are now shown on the website —
https://benchmarksgame-team.pages.debian.net/benchmarksgame/fastest/java-kotlin.html
1 Like
Looking at a few of those it’s no wonder the java versions run faster than the kotlin ones. Whoever ported the java code to kotlin wanted java to be faster. If you copy/convert the exact code from java to kotlin but limit kotlin to use 1 thread while java is allowed to use every core on the cpu it’s no wonder the java version is faster. In another example the java and kotlin code again look pretty much identicall but kotlin uses BigInteger while java uses int.
6 Likes
-
Please be specific – say which programs you’re talking about.
-
Just contribute better Kotlin programs. Sounds like that should be easy for you.
1 Like
Yeah. Benchmark game was always too opionated. It does not use JMH for JVM-based tests and the choise of problems gives a lot of advantage to lower-level languages like C. In this case the results a laughtable, because the performance of kotlin program should be exactly the same as for java program excluding some rare border cases with lambdas.
People probably will fix that in no time. But the project itself is not fair to JVM from the beginning.
2 Likes
It does not use JMH for JVM-based tests…
1 Like
Glad to hear about it. JMH could actually demonstrate a minor slowdown because there are some JVM optimizations that work differently in micro-benchmark environment.
But the project itself is not fair to JVM from the beginning.
On the contrary.
I’m seriously doubting the validity of some of these results in regards to the environment they were run on, take for example the binary-trees benchmark:
| source | secs | mem | gz | busy | cpu load |
|--------|-------|---------|-----|-------|-----------------|
| Java | 8.32 | 953,620 | 835 | 28.37 | 95% 87% 81% 77% |
| Kotlin | 20.97 | 6,656 | 769 | 64.29 | 78% 81% 75% 72% |
The Kotlin version is a near 1 to 1 translation of the Java version, but is 2.5 times slower and uses 99% less memory?
Edit: In fact this seems true for all benchmarks, they all seem to be using about the same amount of memory no matter the test. Is there some kind of aggressive GC going on?
2 Likes
Is there some kind of aggressive GC going on?
Good question.
The build and run options for each program are as-shown in the program log.
However, the as-shipped kotlin-compiler-1.3.61/bin/kotlinc file includes this –
[ -n "$JAVA_OPTS" ] || JAVA_OPTS="-Xmx256M -Xms32M"
I suppose we need to prevent that being default, and set the Java default instead.
Is there some kind of aggressive GC going on?
Now, the initial settings for the Kotlin programs are the same as the default settings for the Java programs -Xmx984M -Xms32M
same amount of memory no matter the test
That was still a problem, so I’ve changed the way the programs are run.
Now, instead-of –
kotlin -classpath fannkuchredux.jar fannkuchredux 12
– the command line is something like –
java -classpath ./fannkuchredux.jar fannkuchredux 12
That seems to give comparable memory-use measurements.
https://benchmarksgame-team.pages.debian.net/benchmarksgame/fastest/java-kotlin.html
Great! The test results are starting to look a lot more like one would expect from a Java → Kotlin translation.
On the topic of kotlin-like code, are external dependencies allowed at all? I’m sure some people would like to rewrite some of the parallel benchmarks with coroutines. While the stdlib supports them, the actual, official implementation is delegated to a dependency to reduce build size. A similar thing with Kotlin’s atomic data types.
1 Like
…external dependencies allowed at all? … official implementation…
Should be OK. (Along with the program source, provide compile / run command-lines that don’t require maven.)
1 Like
The main optimization that allow languages to beat others on the benchmarckgame is about using SIMD.
If someone implement AVX~256 to a kotlin algorithm it should be multiple time faster.
Sadly kotlin nor Java have first class support for explicit SIMD.
But there are some libraries (might be useful GitHub - JetBrains-Research/viktor: Efficient f64-only ndarray in Kotlin)
The real solution would be to use the Vector API
https://openjdk.java.net/jeps/338
Being the same API, one could contribute both for kotlin and Java at the same time 
I invite contributors to take inspiration from the c++ or rust algorithm implementations
BTW off-topic: OpenMP support for Kotlin/kotlin native would be really nice
What about graalVM and ZGC/Shenandoah impact on performance on those benchmarcks?
From May 2019 —
https://benchmarksgame-team.pages.debian.net/benchmarksgame/fastest/java-substratevm.html
Straightforward “idiomatic” Kotlin programs that help show why someone might prefer Kotlin over Java would be wonderful !
Hopefully there’s someone with enough enthusiasm to contribute Kotlin programs that improve on those currently shown.
He probably meant JIT mode.It is much faster on some large math problems due to better escape analysis, but I do not think there will be difference for such small examples with primitives. Substrate currently is not meant for speed. Especially not in numeric tascs.
1 Like
It’s been a while since I looked, so —
looked into the site, there are two java-programs for pi and the first one is compereable, but when you’re cchecking, it checks against the fastest (the spiedier) ont.
| source | secs | mem | gz | busy | cpu load |
|---|---|---|---|---|---|
| Java-2 | 3.07 | 38.408 | 938 | 3.39 | 0% 8% 99% 3% |
| Java-1 | 19.52 | 201,292 | 800 | 21.13 | 2% 57% 45% 4% |
| Kotlin | 20.97 | 6,656 | 769 | 64.29 | 78% 81% 75% 72% |
Although it seems that kotlin somehow traded memory for cpu-intensivity…