Performance Problem


#1

windows 10 x64 1709
java version "1.8.0_152"
Kotlin version 1.2.10 (JRE 1.8.0_152-b16)

kotlin code:

import java.util.*

var sum = 0L
val x = 9000000000L

fun main(args: Array<String>) {
    /*first*/
    sum = 0
    val s1 = Date().time
//    (0..x).forEach(::getSum)
    (0..x).forEach {sum += it + 100}
    val e1 = Date().time
    println(e1 - s1)

    /*second*/
    sum = 0
    val s2 = Date().time
//    (0..x).forEach(::getSum)
    (0..x).forEach {sum += it + 100}
    val e2 = Date().time
    println(e2 - s2)

    /*thrid*/
    sum = 0
    val s3 = Date().time
//    (0..x).forEach(::getSum)
    (0..x).forEach {sum += it + 100}
    val e3 = Date().time
    println(e3 - s3)

    println("----divider----")
    calc()
    calc()
    calc()
}
//fun getSum(value: Long) {sum += value + 100}

fun calc() {
    sum = 0
    val s = Date().time
//    (0..x).forEach(::getSum)
    (0..x).forEach {sum += it + 100}
    val e = Date().time
    println(e - s)
}

result:
8583 // much faster than other same codes
21016
21062
----divider----
20985
20297
20969


#2

java code:

import java.util.Date;
import java.util.Iterator;

public final class Performance {
    private static long sum;
    private static final long x = 9000000000L;

    public static final long getX() {
        return x;
    }

    public static final void main(String[] args) {
        sum = 0;
        Iterator<Long> it1 = new Iterator<Long>() {
            private long value = 0;
            @Override
            public boolean hasNext() {
                return value <= getX();
            }

            @Override
            public Long next() {
                return value++;
            }
        };

        long s1 = new Date().getTime();
        while (it1.hasNext()) {
            sum += it1.next() + 100;
        }
        long e1 = new Date().getTime();
        System.out.println(e1 - s1);


        sum = 0;
        Iterator<Long> it2 = new Iterator<Long>() {
            private long value = 0;
            @Override
            public boolean hasNext() {
                return value <= getX();
            }

            @Override
            public Long next() {
                return value++;
            }
        };

        long s2 = new Date().getTime();
        while (it2.hasNext()) {
            sum += it2.next() + 100;
        }
        long e2 = new Date().getTime();
        System.out.println(e2 - s2);



        sum = 0;
        Iterator<Long> it3 = new Iterator<Long>() {
            private long value = 0;
            @Override
            public boolean hasNext() {
                return value <= getX();
            }

            @Override
            public Long next() {
                return value++;
            }
        };

        long s3 = new Date().getTime();
        while (it3.hasNext()) {
            sum += it3.next() + 100;
        }
        long e3 = new Date().getTime();
        System.out.println(e3 - s3);


        String var24 = "----divider----";
        System.out.println(var24);
        calc();
        calc();
        calc();
    }

    public static final void calc() {
        sum = 0;
        Iterator<Long> it = new Iterator<Long>() {
            private long value = 0;
            @Override
            public boolean hasNext() {
                return value <= getX();
            }

            @Override
            public Long next() {
                return value++;
            }
        };

        long s = new Date().getTime();
        while (it.hasNext()) {
            sum += it.next() + 100;
        }
        long e = new Date().getTime();
        System.out.println(e - s);
    }
}

result:
11355 // faster again…but the codes are all the same…
38782
36432
----divider----
36380
36026
39718

…I don’t know why is that. everything is the same…


#3

I’ve tried your code in the kotlin online, simplifying it to be more compact:

import java.time.*

var sum = 0L
val x = 900000000L

fun benchmark(action: () -> Unit): Duration{
    val start = Instant.now()
    action()
    return Duration.between(start, Instant.now())
}

fun main(args: Array<String>) {
    //first
    println(benchmark{
            (0..x).forEach {sum += it}
    })
    
    sum = 0
    //second
    println(benchmark{
            (0..x).forEach {sum += it}
    })

}

Indeed, if you run it that way, then the output would be like:

PT0.769S
PT2.061S

I think that for such simple operations, compiler (not kotlin, but underlying JVM) does a lot of optimizations, so the actual order of operations could not be the same as declared order. If I use random numbers instead of summing arithmetic progressions, I get the same times.


#4

In addition to the correct observation of @darksnake: Output the sum. The JIT compiler is very aggressive and if it sees that the value calculated inside the loop is not used after the loop, it may eliminate the whole loop, eliminate the statement in the loop, or make some other optimization. By outputting the sum, you are doing something productive that the JIT compiler cannot eliminate.

Microbenchmarking is very tricky. If you want to have meaningful results, you must learn how to use JMH (Java Microbenchmark Harness).


#5

In fact, I tried to output the sum and it does not change the resulting time. I think that compiler sees the same bytecode and reuses previous result. It is a bit strange that fast one goes first.


#6

Weird indeed. The JVM is a very complex beast, so it is hard to say what would cause that.

I stand by my advice to use JMH, and only judge performance based on its output.


#7

Thanks for your reply.
I cannot figure out why the first one is faster. I think the second one should be much faster if the compiler has optimized it. But the result is reversed.

I want to ask you one more question.
It’s well known that reflection could cause a performance problem.
Can I use function reference or property reference without caring about the reflection performance problem?
Just like that.

fun a(value: Any) {/*...*/}
(0..100000).forEach(::a)

#8

You have to run your tests in JMH. These results are not representative of real-life situations.

I think Kotlin will compile this to a direct invocation of a(...), so there will be no overhead compared to: (0..100000).forEach { a(it) }. But to be sure you would have to check the resulting bytecode, or write JMH benchmarks.


#9

Thank you!