Dispatching calls to instance extension functions

[msevcenko]

I’d like to discuss the controversial practice of using instance extension functions.

One possible use case might be, when some argument of a method is in some sense fundamental to the operation, one might be tempted to declare it as receiver of the instance method. Doing so may save some boilerplate and arguably make the code easier to read.

Examples of such usage - e.g. when the method actually represents some operation to be applied to some object. Another example - the fundamental argument is some “context object”, such as a writer to which something is being written. Then you may have a family of related methods with this context parameter, and making it a receiver of these operations, eliminates the need to pass this context among them.

Let me demonstrate with an example:

interface Writable {

    fun Writer.write()

}

class Node(val title: String, val next: Node?) : Writable {

    override fun Writer.write() {
        writeTitle()           // ok
        next?.write()          // error
        next?.run { write() }  // ok
    }

    private fun Writer.writeTitle() {
        write(title)
    }

}

This code demonstrates two interesting points.

1. you can call instance method with the same receiver, on the same instance (the writeTitle method); the receiver object (instance of Writer) is automatically dispatched to the method. In other words, the writeTitle method (or its body) has two implicit receivers, one is Node instance, and one is Writer instance, and they can be both correctly dispatched from a code block, that also has these two receivers.
   2.you cannot call instance method, on different instance, even if it is the same type as the calling method. So you cannot call next?.write(), even if both the calling block, and the called block, have the same receivers. Namely the calling block has the Writer instance as receiver implicitly, and the dispatch command enforces the Node receiver to be the content of the next variable. But I yould really expect, that the Writer instance would be propagated to the method, in this case.

My first question is - is there some fundamental reason why the method cannot be called directly? Instead, you need to use rather cryptic dispatch: next?.run { write() }, which creates a code block, where both instances (Writer and Node) are as implicit receivers, and only then you can call the write method.

Related question is - is it a bad idea to use this practice? Apparently reason for not using it is this problem with dispatching calls, but again, is this a fundamental problem? That cannot, or shouldn’t, be fixed in the language?

[kyay10]

The fundamental reason is simple: the dot syntax is only used for the “explicit” receiver of a method. Typically that receiver is the only one, but member extension functions show the difference there where the extension is the “explicit” one. This design choice is even more important when considering the upcoming context parameters feature, where one can do:

context(Writable)
fun Writer.foo()

The magical incantation of run or with is not that magical really; it simply introduces a receiver into the scope.
The face that both Writer and Node in your example are implicit maybe does show that this decision is a little arbitrary, but it’s consistent with how this functions if Writer was mentioned explicitly instead.

Using this practice is not a bad idea at all, in fact context parameters solidify this concept and expand it. You will however run into issues if you e.g. want a function reference to such a method, or if you want to call the super method of it. These issues should be solved eventually though.

[msevcenko]

@kyay10

The fundamental reason is simple: the dot syntax is only used for the “explicit” receiver of a method. Typically that receiver is the only one, but member extension functions show the difference there where the extension is the “explicit” one. This design choice is even more important when considering the upcoming context parameters feature, where one can do:

I am not sure if I follow this argument. So it is not something truly fundamental, it is a design decision?

I mean, the language designers have had these two options (try propagate the contextual receiver or not), and they deliberately decided not to, for some reason. So my question was, what was the rationale behind this decision, if any has an idea. Maybe it is really something fundamental that I’ve overlooked, maybe the other way would be more difficult to implement, maybe it would just be more confusing (than it already is ), to which I would argue, that it’s perhaps more natural than the current state.

So the reason why it does not work is really simple - it was designed so, but that is not that fundamental reason I am looking for.

The magical incantation of run or with is not that magical really; it simply introduces a receiver into the scope.

Well, I didn’t mean it is completely incomprehensive, it is just cryptic and ugly. I think there is no question which of these two statements are clearer:

        next?.write()
        next?.run { write() }

Actually, since I already use this practice, I “invented” an arguably slightly less ugly syntax, by redefining the “Any.invoke” same as the Run scope function, so I can omit the run call and write e.g. next { write() } instead of next.run { write() }.

You will however run into issues if you e.g. want a function reference to such a method, or if you want to call the super method of it.

True, unfortunately. The first actually is not an issue, just do not use this syntax, inability to call super method, on the other hand, is pretty bad. And actually is a consequence of the arbitrary design decision discussed. If the propagation of the context parameter were implemented, you could just use normal super.call() syntax.

This design choice is even more important when considering the upcoming context parameters feature

Maybe it is related to this feature. I am not familiar, so I have a look what it is, thanks for this reference.

[kyay10]

Let me try to go further into that first question then. It is a design decision yes, but one motivated by a simple idea: Every specific function should have one possible type that goes before the dot when using dot syntax. Based on that, Kotlin always designates the extension receiver (if exists) as the explicit one. If not, then the dispatch receiver, and if not, then no explicit receiver exists. This is especially important if the dispatch and extension receivers could have the same type (this shows up in DSLs that use sealed hierarchies and such). Imagine we have:

interface Element
interface CompositeElement: Element {
  fun Element.addHere()
  operator fun Element.unaryPlus() = addHere()
}
with(someCompositeElement) {
  +someOtherCompositeElement
  someThirdCompositeElement.addHere()
}

What should happen to those calls? The thing that makes the most sense is that the explicit receiver there is referring to the extension receiver for the methods. Conceptually then, CompositeElement is seen as a “scope” that is intended to be used implicitly like that.
Context parameters make this separation a lot clearer btw. In fact, I think it’d be reasonable to deprecate instance extension functions and replace them instead with context-based alternatives (although one issue is that you need to import extensions explicitly, while instance methods are automatically included when importing a class).

[msevcenko]

@kyay10

Let me try to go further into that first question then. It is a design decision yes, but one motivated by a simple idea: Every specific function should have one possible type that goes before the dot when using dot syntax.

I am not sure if I understood it correctly.

In other words, this simple idea just says that you should not dispatch two receivers at the same time, even if it were possible. Because it is simpler.

Then you demonstrate that there may be some ambiguity, that if you can dispatch by explicit receiver and by receiver defined by the class of that method, you prioritize the explicit (extension) one. Is that right? Ok, but I didn’t consider this ambiguous case. After all, it is already ambiguous that inside the extension instance method, the “this” reference is already ambiguous, may mean two different things, and is actually defined to prefer the explicit receiver, not the instance of the method.

So again, I would argue that you could design the dispatch to support direct call e.g.

next?.write()

There is no ambiguity here, is it? You have the implicit instance parameter, and the explicit receiver, and they are of different types. And there is no method with the same name, that would accept only receiver or only instance, there is just this one method that requires both. So either you can dispatch it, or report an error.

The same is with calling the super method. It is clear that if the method has a receiver, and the super method has a receiver of the same type, that it is intended to pass it from the subclass to the super implementation. Where is the problem?

So I think conceptually there is no problem here. It may introduce some new nasty ambiguities, which you can easily avoid. And it may be more complicated to actually wire the callers correctly.

[kyay10]

I mean theoretically yeah you could have next?.write() work by using the types to figure it out.
However, this would actually result in ambiguities and clashes in pre-existing code.
For instance.

fun Writable.write() = TODO()

next?.write() would previously resolve to that extension, but now it would resolve to the member extension with your proposal.
Also, what you want could (I think) be achieved with:

context(w: Writer)
fun Writable.write() = w.write()

So it’s not like it’s impossible to have this work, it just might introduce issues in pre-existing code.

Another question is how does your design handle operators? Operators right now use that same “explicit receiver” idea. If we had:

interface Writable {
    operator fun Writer.unaryPlus()
}

Then would +(node!!) work? I don’t know if I like that very much personally. Maybe you’d want your design to only work on non-operators then, but how do you handle node?.unaryPlus() then?

[msevcenko]

@kyay10

I mean theoretically yeah you could have next?.write() work by using the types to figure it out.
However, this would actually result in ambiguities and clashes in pre-existing code.
For instance.

Yes, doing it retro-actively could be a breaking change.

Maybe it could be remedied by (a) enabling it with compiler flag and (b) supporting only cases where it would be not ambiguous, or it would have the least precedence, so it would not change existing code. I believe this is possible.

My question was more conceptual, I don’t really expect someone would be actually fixing it.