The Mystery Of The Misleading, Multiplying Meta-Methods

I suspect there’s a deep omission in Kotlin. (That shouldn’t be a surprise, of course. No language is perfect, and even though Kotlin fixes many problems and warts from many other languages, it’s bound to hit the occasional new one. And I still love it regardless.) But I haven’t seen this discussed or even hinted at, and it worries me.

I’m afraid I don’t have a solution, nor even a clear definition of the problem; it’s just something which my gut tells me isn’t right, an area of ugliness which suggests to me that a different approach might be needed.

The main symptom is the number of methods that don’t do anything but control how another block of code is run: run(), let(), apply(), with(), also(), takeIf(), &c.

Now, all of these do something useful; and it’s a great reflection on Kotlin that you can do all these things in plain methods, when in many other languages they’d have to be new language constructs (or impossible or meaningless).

But they have problems: their exact function is confusing; they’re very similar to each other; the differences between them are subtle; and their names don’t tell you enough about what they do. (This is surprising, as Kotlin’s use of English is exceptionally good on the whole.) It’s hard to understand exactly what they all do, and which ones should be used where. Their names don’t give much clue as to whether they pass their receiver as ‘it’, or ‘this’, or not at all; whether they always call their block; and whether they return their receiver, or their block’s result, or something else.

For example, other posts hereabouts have recommended using let() for method chaining, for avoiding temporary variables, for conditionally executing code, and for various other purposes that aren’t really related to its direct meaning. That’s a lot of conceptual weight for such a little method! And similar applies to all the other methods. (What’s the difference between also() and apply()? I bet many of you don’t know without looking it up!)

This worries me because these methods, useful as they are, make the language a little more obscure, harder to learn, harder to read, and more error-prone.

As I said, that’s just a symptom; it suggests to me that something is missing in the language, some better way of doing all these things.

But I don’t know what that missing thing should be!

It could be new language feature(s), rearranging existing language features, and/or simpler and more general library methods. But Kotlin’s mature enough that new syntax is unlikely, and so none of this will get fixed unless someone can come up with a new scheme which is obviously better.

Does anyone have any ideas? What could go in place of all these methods, that would be clearer and more readable? How could the language be tweaked so as not to need them?

3 Likes

I’m in a hurry, so I will post link later, but there is a post suggesting the syntaxis:

a.let{ this->...}

This is great for some functions, but I think I will zich to the original.
When using them a lot, you will find when to use which.
I believe this topic is also mentioned in the guidelines.

You need to remember just 4 (four) functions (not methods). Is that difficult?
Google “Kotlin scope functions”, there are some useful guides, for example https://kotlin.guide/scoping-functions

1 Like

This is what run() function do.

1 Like

Yes, but in this way, you can use this with other functions as well.
And I completely agree with those functions, they are a second nature to me

I have to agree with the others here, I actually think the way Kotlin does this is superior to any dedicated syntax I’ve seen so far. Yes, you have to get used to the function names first, but for me that didn’t take long, and yes, I can tell you exactly what each one does without looking it up, just because I constantly use them because they are so useful.

1 Like

I don’t find them ugly. And I would completely expect there to be a plethora of them.

The names are only vaguely mnemonic. apply is roughly what some other languages like Ruby call tap, but that name isn’t all that much better. In language communities more friendly to symbolic operators, the escape hatch would be to give up on naming and pick a symbol. A convention could then encode if there’s a receiver and if the block’s return value is ignored. (And come to think, let resembles pipe |>, though with none of the special sauce based on context, like list or optional.)

What I found most challenging while learning to use these in Kotlin is how KDoc does not group them together, and there are not cross-refs to cover for that omission. Breaking the many functions at kotlin - Kotlin Programming Language into task-based groups would make it easy to quickly refer back to the docs while learning the lingo. Without that, generous use or cross-refs, or tagging with something reliably searchable (“this is one of Kotlin’s scoping utility functions”) would go a long way.

(I encounter similar challenges with Collection-alikes having scads of extension methods, so that it can be hard to see what methods are unique to that type vs are part of the common aggregate manipulation language.)

ETA: For what it’s worth, Lodash calls also tap and calls let thru. These names are not all that much more helpful. That its documentation groups them in a section called “Seq” Methods, though, is quite helpful.

ETA2: It turns out that the Coding Conventions document, near the end, outlines a quick decision tree for selecting a scope function. This is pretty much what I wanted to see in KDoc. I wonder if we could get back-links added from the function doc comments to this Conventions section?

3 Likes

I find myself completely at ease with them after discovering this gist. It all makes sense when you see it laid out

There is a fundamental lie in object oriented programming. The lie is that functions or methods are intrinsically properties of their enclosing objects. Say I have a class

data class Foo(val bar: Int, val baz: String) {
    fun combine(other: Foo): Foo {
        return Foo(this.bar + other.bar, this.baz + other.baz)
    }
}

and I want to use the ‘combine’ method. I might use it like this

fun main(vararg args: String) {

    val one = Foo(1, "one")
    val two = Foo(2, "two")

    // Common invocation
    val three = one.combine(two)
    println(three)
}

This should be fairly familiar. Foo owns a ‘combine’ operation, and so we use the dot operator on an instance of Foo in order to invoke it.

It’s not necessarily a ‘problem’, but the ‘thing’ is, there’s no reason to do it this way. Access modifiers on properties of data types, and subsequently intrinsic methods to operate on private properties, are just a superfluous idea from the OO world. If implementation or state needs to be hidden, that should be done using interfaces (or type classes in more functional languages).

With just as much validity, we could implement the above as the following:

data class Foo(val bar: Int, val baz: String)

fun combine(some: Foo, another: Foo): Foo {
    return Foo(some.bar + another.bar, some.baz + another.baz)
}

fun main(vararg args: String) {

    val one = Foo(1, "one")
    val two = Foo(2, "two")

    // Pure external function instead of instance method
    val three = combine(one, two)
    println(three)
}

And now we realize the first glimpse of the problem you are worried about. Why are there two different ways to implement and invoke a ‘combine’ function? How do I choose between the two? It is this seed of uncertainty that leads down the path to the helper functions we have today, and you mention in your post.

You can see from the above example that instance methods have questionable purpose. In OOP it is touted from private state management. For languages with weak support for function composition, it is (half as) useful for ‘fluent’ programming and chaining method calls. You can think of instance methods as a strange pattern of organizing functions for infix invocation.

this.combine(another)

compared to

combine(this, another)

There really isn’t a difference.

BUT. It’s the world we live in. We have this strange notion of ‘this’, the current instance of a class or object enclosing a method. So how do we cope in this world? How do we cope in a world where we really like ‘fluent programming’?

Sometimes (very very very frequently, actually), a class doesn’t define all the methods that would be useful for it to have. What if we could define our own methods on other people’s classes? Enter extension functions.

Why isn’t creating a normal function that just consumes the class as an argument good enough? No real virtuous reason, people just really like and are familiar with using dot operator aka infix invocation they are used to with OO methods.

So you’ll notice that whenever you define an extension function, you are actually just defining a normal function, with the only magic part being that the invocation target is an implicit first parameter. This implicit first parameter is named ‘this’.

data class Foo(val bar: Int, val baz: String) 

fun Foo.reverse(): Foo {
    return Foo(-this.bar, this.baz.reversed())
}

In the above sample right here, the function reverse() is NOT a function with zero parameters… it is a function with one implicit parameter, that gets called ‘this’. The fact that reverse() is an ‘extension function’ rather than a plain function is just sugar.

So now we have this notion of an implicit parameter ‘this’. But you will notice something interesting. Any time a block of code has a contextual ‘this’, you don’t have to actually ever say ‘this’. All the properties of the object are implicitly brought into scope as well.

data class Foo(val bar: Int, val baz: String) 

fun Foo.reverse(): Foo {
    return Foo(-bar, baz.reversed())
}

That detail is sugar as well. It is just pure sugar. If you like the taste of it, then it’s good as far as you like it. But that being said, none of it is really real. When you boil it down, what we have is still just a data type and a function with normal arguments.

Now, keep all that in mind, and we shift topics slightly.

Lambda functions are useful because it lets us introduce ad-hoc functions, which is useful for any language supporting higher order functions (functions are literally data, just like any other piece of data). Lambdas, similar to instance methods, are ‘just functions’ as well, no matter how much sugar we sprinkle on top. One particularly useful piece of sugar however, is an implicit first parameter. Sound familiar? Just like how instance methods have an implicit first parameter of ‘this’, lambdas (can) have an implicit single parameter of ‘it’. You can rename the parameter if you want, but the implicit parameter is always there for your convenience whenever you want it.

So all that being said, you’ll notice we have a few different ideas about how to dump sugar on our functions:

  • There are plain functions with no implicit arguments
  • There are ‘instance methods’ that are really just functions with an implicit first argument, called ‘this’. Every property of ‘this’ is also implicitly brought into scope.
  • There are lambdas with single arguments, where we can have an implicit argument called ‘it’ if we so choose
  • And lastly there are extension functions, which are just knockoffs of instance methods

So, finally, to address your concerns, we hone in on the extension functions.

At the very very simplist level, we would just have plain data and pure plain functions. However, we now have all this sugar at our disposal. You are writing a function for your project. Or perhaps you’re not writing a function directly, rather you want to take an object and do stuff to it. So you decide to use a function or lambda of some kind. With all these different types sugar for your function or lambda, how do you choose?

The choice is up to you. But by now, hopefully the usefulness of that gist I provided at the top of this post is starting to show. Here is how I walk through the choice:

Will I be feeding my object to a named function, or a lambda?

  • With a named function, just use it, nevermind all these extension functions. These extension functions are at least 90% intended for use with lambdas.

Do I want implicitly to bring the properties of the target object into the scope of my lambda?

  • If yes, then use one of the extension functions that passes the target as ‘this’ (apply or run or with).
  • If no, then use one of the extension functions that passes the target as ‘it’ (also or let).
  • This is an important decision. I recommend heavily leaning towards ‘it’ instead of ‘this’ options, unless your lambda is specifically for initializing properties, or something else where implicitly bringing stuff into scope is useful. (you never actually have to type ‘this’, but still have to type ‘it’).

What do I want the return value of my lambda to be?

  • The target (also or apply)?
  • or the lambda return value (let or run or with)?

How do I want to invoke my lambda?

  • dot operator / fluent style? Use the extension functions (also apply let run)
  • normal function invocation? Use with to pass the target as ‘this’, or just use the invoke operator () directly on the labmda, feeding the object as an argument there (if you want the target passed as ‘it’)

And with all that in mind, you can actually see the method (pun unintended) in the madness. All these different choices are really just the different combinations of sugar. The more varieties of sugar we have for function definition, then combinatorially the more helpers we need for function invocation (to achieve equal helpfulness for all possible combinations of sugar). While there may be a lot of options, this isn’t a slippery slope. All the combinations are covered. The only way we’d add more is if we invented even more sugars for function definition (which to be fair, may happen, but still isn’t a huge slope even if it is a slippery one).

Hopefully that addresses most of your concern

You may notice the outlier takeIf. The idea with takeIf and takeUnless are to evaluate predicate that determines an already fixed specified value. The the others have a slightly different intention, which is to perform operations on (or otherwise consume) the target. Without 100% pure functional and immutable guarantees, someone could still do something stupid and funky inside takeIf (like mutate a property), but I think more experienced coders would cry foul and recommend the other helper functions in such a case.

At this rate, you may notice the semantics of the names of these functions starts to fit into place.

  • also - return the target, but also do this thing to it
  • apply - return the target, but apply some operations to it first - these operations borrowing the target’s internal scope (implicit this)
  • let - let the target (it) be transformed by this lambda into the result of the lambda (this is kind of like the map function, but on a single object instead of a collection)
  • run - Take this object and borrowing its internal scope, run this lambda, returning the result of the lambda
  • with - With a normal function argument, borrow its internal scope (implicit this) and return the result of the lambda

Granted, englishifying how they work is very contrived, and maybe some of the function names work better than others, but the semantics fit at least enough for me to tell them apart.

tl;dr:
These helper methods are an artifact of OOP and sugar to support OOP. That’s not to insult, Kotlin is actually my favorite OOP language I’ve ever used and I think they are super useful as far as OOP goes. It just inflates the need for a helper-function vocabulary a bit. If you want a more cohesive, trim, and refined experience, I recommend giving an ML family language a try (e.g. haskell, purescript, Standard ML)

5 Likes

Another fresh article: Kotlin Scope Functions

1 Like

Sorry, @abrownvt.

In your very long post you laying down the argument in “religion” war between OOP and FP. The original post was about naming of specific functions. Please do not hijack the topic.

If you personally see no use for OOP you should use pure FP language. There are plenty of them around.

Kotlin is the language fussing together both approaches and allowing developer to choose the best one for each specific task (on the function/method level). I see this very helpful and actually using both.

@cabman if you actually read my post, you’d know I said OOP’s way of doing things is not problematic, it is just a flavor of magic, a sugar. I also specifically said that I meant no insult in referring to sugar, and that Kotlin is my favorite language for OOP. I never said there is no use for OOP, and I do also use OOP just like you. The deconstruction of sugar isn’t a religious war, it is just explaining how things work. I just explained the facts. Take it easy out there.

In regards to ‘hijacking the topic’, the question (as I understand it) is not just about how these functions are named (‘misleading’ in title), but why there are so many of them and if there will be more (‘multiplying’ in title). I hope that helps piece together the relevance for you and and put your concern at ease.

One practical reason is to allow programmers to leverage autocomplete as a way to query for available operations upon an instance of a type.

One theoretical reason is that object.message clarifies the code we’re trusting to properly perform the operation requested. If theirObject is not trusted, you probably want to prefer myObject.combine(theirObject) to theirObject.combine(myObject). I believe The Left Hand of Equals has some interesting things to say along these lines.