Self Types


#21

Yes it will have erased types in addition to generic types (as the case will all other generic methods). Take the following example:

open class KtParent { // Class in Kotlin with self-typed property
    val child:Self
}

class KtChild: KtParent()

The function KtParent.getChild():KtParent is generated on KtParent
A synthetic function KtChild.getChild():KtChild is generated on KtChild

class JavaChild extends KtParent { }

No KtChild KtChild.getChild() is generated in Java.

In the case that KtParent is created with a generic parameter representing the self type you are in recursive generics land with all the major issues that entails (beyond the fact that recursive generics cannot express the full semantics of a Self type). The problem here is never runtime, it is always compile time and correct code being generated from Java.

Method bridging would be the expected implementation for covariant return types, but indeed it may be possible to replace it with checkCast at the call site instead. Of course all this (with or without bridging) means that the contract can be violated in Java without warning (there is nothing in Java to allow specifying meaning). Synthetic methods would be needed for correct usage from Java (the Java compiler does not understand it needs to inject a checkCast and treat the value as a subtype).

The difference with declaration site variance is that it is more of a syntactic sugar that removes the need to specify variance at use site. The variance is still specified at use on a binary level, so it is impossible for Java to violate Kotlin classes, even if it is able to use the type itself with different variance.


#22

Are we talking about JVM interopability or Java interopability? If it is just JVM interopability, it is not really that complicated, since everything is just java.lang.Object with unchecked type casts. As for the List example, it is even easier, because lists don’t contain type in the runtime anyway, so we don’t need to care about the List T type at all.

Looks like what really matters is the Java interopability. But are there really that many people calling Kotlin from Java (with the API also in Kotlin)? It isn’t really a big problem if the generated class files force Java context to do unnecessary casts as if there were no generics.


#23

The problem is that call from java is allowed. This means that someone could call a kotlin function from java and do something that is prohibited in kotlin, thus creating an undebugable error.

I think that the solution to those problems is to add a compiler flag that somehow forbids to call kotlin code from java (package name mangling or something). In this case one can implement some kotlin-only features without fear of backward compatibility problems.


#24

Sounds like the same solution to many other problems like inline functions accessing private methods.


#25

A synthetic function KtChild.getChild():KtChild is generated on KtChild

I contend there is no need for synthetic methods and the like. The function on KtParent can be generated to include type information:

KtParent.getChild():KtParent <~~~ compiler annotates with self

As such the compiler can always infer the self type; there is no need for synthetics.

As for Java, again it can override methods employing self from Kotlin classes. From the Java perspective there is no self type, therefore Java code must cast where necessary.


#26

I note the Manifold project for Java advertises self types:

http://manifold.systems/docs.html#the-self-type

I wonder what they are doing under the hood to make this work?

They also say they support non-Java JVM languages (with some help):

http://manifold.systems/docs.html#use-with-other-jvm-languages


#27

I wonder what they are doing under the hood to make this work?

Hi, I’m the author of Manifold. As a Java compiler plugin Manifold overrides javac’s attribution phase and infers the type at the call site via the @Self annotation available from the method reference’s declared return type. Likewise, the Manifold IntelliJ plugin overrides IJ’s type resolution to implement @Self.

They also say they support non-Java JVM languages (with some help):

With lots of help! :wink: Seriously, I should provide more information regarding implementing a Manifold host, it can be pretty involved depending on the complexity of the hosting language’s type system. Regarding Kotlin I’ve started and back-burnered a project for that. I’ll start it back up soon, but Manifold is still a pre-1.0 project and I’d like to button that up first.

Manifold is essentially two categories of features:

  1. Type Manifolds: a foundation for extending a language’s type system to include new domains of types such as JSON, Javascript, DarkJava, CSV, Templates, SQL, etc.
  2. Extensions: extension methods, structural typing, type-safe reflection, self type, etc.

The Kotlin Manifold host is exclusively #1.

Read more about Manifold here if you’re interested.


#28

In some cases you are probably right. It is however inconsistent with how the Java compiler implements covariant return types (or generic return types with a more specialised or concrete generic value). But as you explain in the manifold post, it uses a compiler plugin, also at use site. Of course self-type return values are the easy case. Self-types can be parameters for functions or even parameters to a generic type.


#29

The use case for having a Self type for a return type could be solved in the same way Dart does it with the cascade operator.
Vector()…set(1, 2)…add(2, 3)
(this is meant to be 2 dots, not 3, I don’t know why discourse changes two dots to 3)

The only other use case I frequently hit is for a callback that passes a reference to self.
e.g.
val myComponent = Button() // extends UiComponent
myComponent.onActivated { it as Button } // Would be better if the cast wasn’t needed.


#30

You get no argument on the desirability of a Self type. The problem is doing it while maintaining Java compatibility. In the meantime you can still use the recursive generic pattern.


#31

You get no argument on the desirability of a Self type. The problem is doing it while maintaining Java compatibility. In the meantime you can still use the recursive generic pattern.

What about Any then? It doesn’t exist in Java, but we do have it in Kotlin. My point is: if we have a dedicated Self type in Kotlin, and the Kotlin compiler is aware of it, that’s enough for a first step. If somebody wants to use the same API from Java, I’m okay with the return type Self being translated to Object. This way we could enjoy the advantages of Self at least as long as we stay in the Kotlin world.

The recursive generic pattern is extremely ugly. I’ve worked with it for a long time, on several different occasions. It adds a lot of complexity to the classes, and it’s a typebound that really doesn’t add any information (e.g. it’s not like List<T> where T actually carries useful information for the caller).


#32

Yes, there is Any in java. It get’s translated to java.lang.Object. The same is true for List and MutableList which both will get translated to List in java.

This would require at least a lot of additional type checks and casts. And I’m sure there are some edge cases that get more complicated.
We also have to consider how usable any API with self types would be from the java side, as this is part of kotlins design philosophy. Is it acceptable that java developers can override functions in a way that they don’t fulfill the contract of a function? How do we guard against this?

Don’t get me wrong. I’d love to see self types in kotlin. Again this is a situation where Kotlin is restricted by java. However this does not mean, that we can just ignore java interoperability as I think it’s one of the main selling points for Kotlin.


#33

We also have to consider how usable any API with self types would be from the java side, as this is part of kotlins design philosophy.

I think Kotlin has parted ways with this philosophy with the introduction of suspending functions. As much as I share the sentiment that it should remain interoperable with Java, I’m okay with Kotlin having a superset of capabilities. The best language to utilize a library written in Kotlin is - of course - Kotlin, as it is the case with pretty much any other language out there. The main thing that needs to be preserved is that a determined Java programmer could, all syntactical barricades aside, utilize a Kotlin library. That property should be retained. If “syntactical barricade” means having to do a ton of manual down-casts… so be it.


#34

They haven’t really. Suspending functions have a well defined signature that can’t be mistaken for anything else. All the suspend keyword does is add a (in kotlin hidden) parameter to the function, at least when looking at the API ( There implementation also get’s converted into a state machine, I think, but that has nothing to do with java interop).
But the important difference to self types is, that this “suspend parameter” can not be mistaken for anything else, when implementing a function with this signature. A self type on the other hand would look (for a java developer) like just another parameter or return type, e.g. there is no way for a java developer to know that he has to return a specific type.


#35

Self types do not necessary break java, they could translate to generic Objects and use casts only when used. It will look ugly from java side, but so do coroutines and extension-based libraries.


#36

Maybe I miss something about the JVM but let’s take this example:

abstract class Foo {
    fun foo(): Self
}

AFAIK the best way to represent this in bytecode would be something like this

abstract class Foo {
    fun foo(): Foo
}

Kotlin would still know this is a self type due to the @Metadata annotation generated by the compiler, but java would have no clue. So how would a java developer seeing this know that he can’t just return any implementation of Foo when inheriting from it?

Even with generics I don’t see how this can be solved.

abstract class Foo<T: Foo> {
    fun foo() : T
}

Nothing but the metadata stops me from creating this implementation, and only the kotlin compiler cares about this metadata.

class FooImp1() : Foo<FooImp2>(){
    override fun foo(): FooImp2 = FooImp2()
}
class FooImp2(): Foo<FooImp1>() {
    override fun foo(): FooImp1 = FooImp1()
}

Pleas correct me if I’m wrong. I don’t care if the implementation from the java side looks a bit ugly. I do however care if there is no real way for java to see that something is intended to be a self type.
Many times when I implement an interface or inherit from another class I let the compiler tell me which functions I need to override. If this can’t tell me about a self type I won’t know about it.


#37

As an explicit cast in place, where it is used.
I am not saying you could use it from java or that it will be useful, I am only saying it will be visible… somehow. As I already said in multiple places, I think that kotlin is past upholding full backward compatibility with java. It is great to be compatible in most things, but self types are usually internal details of implementation. I am not advocating their introduction into the language, just saying that there is nothing terrible with them.


#38

Thx for the clarification. So I guess I am not missing any obvious way of implementing them without breaking some kind of compatibility.
Maybe Kotlin is past upholding full compatibility with java, maybe it’s not. Inline classes are an example that does not work with java so maybe kotlin real does not need to care about this as much as it used to.


#39

Please distinguish the following feature types:

  1. A feature that is visible in Kotlin only, not visible nor usable in Java (e.g. providing named arguments when calling functions and mixing up the order of arguments)
  2. A feature that is visible in Java, but it’s usage is very ugly in java (e.g. extension functions)
  3. A feature that leads to potentially incorrect usages from Java (e.g. non-null types)

Do we agree that Self types fall under point 3?

As far as I understand, there is no easy way to implement Self types in Kotlin without introducing dangerous holes in the API, when used from Java. For example it can happen that the Kotlin code expects an expression to be of a specific Foo subtype but at runtime it is another type because of mixing with Java code.

This can only be prevented by adding automatic runtime checks wherever a self type is used. This would not even solve the problem completely. It would just make sure that the program fails fast at runtime.


#40

I agree that self types would fall under 3. There would be no way enforce that a class extended in Java use ‘this’ where it ought.
I would contend however that there are some cases where you are forced to do type checking because of the lack of this feature. Then your choice is to either add cyclomatic complexity and check for the case that the argument isn’t what you expected, or to trust that the implementation used ‘this’ (as your api describes), and do an unchecked cast. In which case subclasses, both Java or kotlin have the potential to break the api.

As you already pointed out, kotlin already takes a stance that it is “more type safe than java”. Java doesn’t support null types? Screw them, they can be less safe, more reason to use kotlin. They don’t support Self types? Well, then they’re stuck with needing to do an unsafe type cast. Too bad for them.