Valid use case for "disallowed" assign lambda expression to kotlin SAM interface


#1

I have been passing an Int -> Boolean predicate function to another function to process a large number of ints. This performed poorly compared to what I thought was equivalent java code, so I looked into why.

The bottom line was that the kotlin compiled code was using boxing and unboxing of parameters and return values. This happens because kotlin functions are converted by the compiler to instances of the kotlin generic FunctionN interfaces. As a result, each call to the function needs to box any primitive parameter values and unbox any primitive return values. The java code was using a specialised functional interface that involved primitives (i.e. no generics).

Here is a stripped back example of the slower kotlin code:

fun processWithKotlinFunction(numbers : IntArray, predicate : (Int) -> Boolean) : Long {
	var sum : Long = 0
	for (n in numbers)
		if (predicate(n)) sum += n;
	return sum;
}

val numbers = IntArray(10000000, { it -> it })
var predicate : (Int) -> Boolean = { it%2 == 0 }
var result : Long = 0
val nanos = kotlin.system.measureNanoTime { result = processWithKotlinFunction(numbers, predicate) }
println("($nanos ns) processWithKotlinFunction = $result")

Base on the knowledge that generics and boxing/unboxing was the cause of the performance difference,
I improved things by defining a kotlin interface with specialised types and passing it instead of a generic function:

public interface KotlinIntPredicate { public operator fun invoke(n : Int) : Boolean }

fun processWithKotlinPredicate(numbers : IntArray, predicate : KotlinIntPredicate) : Long {
	var sum : Long = 0
	for (n in numbers)
		if (predicate(n)) sum += n;
	return sum;
}

val numbers = IntArray(10000000, { it -> it })
val predicate = object : KotlinIntPredicate {
	override fun invoke(n: Int): Boolean = n%2 == 0
}
var result : Long = 0
val nanos = kotlin.system.measureNanoTime { result = processWithKotlinPredicate(numbers, predicate) }
println("($nanos ns) processWithKotlinPredicate = $result")

As expected primitives are used at runtime without boxing/unboxing, making this example between 3 and 5 times faster! BUT, a clunky object expression must be used.

If I define the interface in java instead, I get the performance improvement AND the ability to use a lambda expression instead of an object expression …

// JavaIntPredicate.java: public interface JavaIntPredicate { public boolean invoke(int i); }

fun processWithJavaPredicate(numbers : IntArray, predicate : JavaIntPredicate) : Long {
	var sum : Long = 0
	for (n in numbers)
		if (predicate(n)) sum += n;
	return sum;
}

val numbers = IntArray(10000000, { it -> it })
val predicate = JavaIntPredicate { it%2 == 0 }
var result : Long = 0
val nanos = kotlin.system.measureNanoTime { result = processWithJavaPredicate(numbers, predicate) }
println("($nanos ns) processWithJavaPredicate = $result")

It is frustrating that lambda style expressions are allowed for java functional interfaces but not for kotlin ones.

I can accept that the normal kotlin style lambda val predicate : KotlinIntPredicate = { n : Int -> n%2 == 0 } doesn’t compile as the types are not compatible. Even if the interface were to extend ‘Int -> Boolean’, it still wouldn’t be valid as we you could argue we are assigning an instance of a supertype to a subtype (so I can understand this being a challenge for the compiler). The compiler error message clearly indicates the types are incompatible: ‘type mismatch: inferred type is (kotlin.Int) -> kotlin.Boolean but Test.KotlinIntPredicate was expected’. Enhancing the compiler to support this is probably complicated and would likely require a range of other things to be fixed/changed as well.

But I can’t see why the java interop style lambda val predicate = KotlinIntPredicate { it%2 == 0 } doesn’t compile. It works fine it the interface is found in a class file built from java. Also, the compiler error message is misleading: ‘unresolved reference: KotlinIntPredicate’. The interface exists and is resolved elsewhere in the code. This made me wonder why the compiler thinks it can’t see the interface?

I thought I could trick the compiler into letting me use the java interop style to assign a lambda to a kotlin functional interface by compiling the kotlin interface separately and putting the corresponding class file in the same place the compiler was told to find the java interface. If it found a class file defining the interface, why would it care whether the class file was originally built from java source or kotlin source. But it still failed with the same unresolved reference error.

It appears as though the kotlin compiler is deliberately ignoring interfaces compiled from kotlin code when processing this type of lambda expression! Maybe there is more going on here, but I can’t help think this particular use case wouldn’t be that hard to support. I can’t see why allowing a java interop style expression to be used would require significant changes to the language, given that it works fine when the interface was defined with java source. Why would any code making use of the resulting interface instance care where the interface definition came from?

The documentation on java-interop says ‘since Kotlin has proper function types, automatic conversion of functions into implementations of Kotlin interfaces is unnecessary and therefore unsupported’.

I respectfully suggest that the documentation is wrong.

Using functional interfaces in kotlin is necessary for primitive specialisation and optimal performance (at least on current JVMs). Converting lambdas into implementations of kotlin interfaces is useful - without it the cumbersome object expression syntax must be used. In fact, concise code for this use case can only be achieved by defining the interface in java - which is a somewhat bizarre situation.

This restriction seems to be arbitrary, gets in the way of legitimate use cases and brings no benefits that I can see. Relaxing the restriction (at least with the java interop style of lambda expression) would not complicate the language from what I can see - it would instead make it more consistent.

Can anyone explain why this restriction exists and/or what I am missing?

The only other solution I can think of (to the generics/primitives/performance issue detailed above) is to have the compiler generate some extra methods for use when a function is called with parameter or return types that are non-null types that map to primitives. I know this will turn out to be unnecessary when the java valhalla value types and associated generics enhancements arrive in the jvm, but that won’t be until java 10 at least (and so isn’t going to apply to kotlin for quite some time).


#2

Thanks for the use case, I agree that it’s rather important.

The best workaround available for you ATM is to define the interface in Java, but we’ll see what can be done about this in the future.

Interfaces can’t be referenced like this. When you say Runnable {}, it’s not an interface you are referring to, but a function (automatically synthesized by the compiler for Java SAM interfaces).


#3

Thank you - a light has now gone on! The compiler is not matching to the java interface while processing the ‘java interop’ style lambda, it is matching it to a function which has been generated based on the java interface. I am still hard wired to need ‘()’ characters to recognise a function call.

As long as an appropriate inline function is defined, I can use the same syntax used for java functional interfaces, but with the interface in kotlin. I think this represents a better solution that avoids having to define the interface in java.

Rewriting the relevant example from above:

public interface IntPredicate { public operator fun invoke(n : Int) : Boolean }

inline fun IntPredicate (crossinline f : (Int) -> Boolean) : IntPredicate {
	val impl = object : IntPredicate {
		override fun invoke(n: Int): Boolean = f(n)
	}
	return impl
}

fun processWithKotlinPredicate(numbers : IntArray, predicate : KotlinIntPredicate) : Long {
	var sum : Long = 0
	for (n in numbers)
		if (predicate(n)) sum += n;
	return sum;
}

val numbers = IntArray(10000000, { it -> it })
val predicate = IntPredicate { it%2 == 0 }
var result : Long = 0
val nanos = kotlin.system.measureNanoTime { result = processWithKotlinPredicate(numbers, predicate) }
println("($nanos ns) processWithKotlinPredicate = $result")

This still requires a bit of verbosity where the interface is defined, but everything else is rosy:

  • the functional interface specialised to primitives is defined in kotlin
  • all code is concise and clear (aside from the single function associated with the interface)
  • using instances of the functional interface performs well (no boxing/unboxing)
  • there is no need to inline the higher order functions just to get this performance!

Ideally I would still like to use val predicate : IntPredicate = { it%2 == 0 } and avoid the inline function, but the above is more workable than what I had before (with object expressions everywhere the functions were defined).