How to protect against misplaced return?

We just had another huge, massive outage costing billions and billions of dollars due to this bug:

fun runTask(): Status {
    withTransaction { transaction -> 
        // ...
        return status
    }
}

which should have been instead:

fun runTask(): Status {
    return withTransaction { transaction -> 
        // ...
        status
    }
}

In this case, withTransaction looks like this (simplified):

suspend inline fun <T> withBatch(
		f: (Transaction) -> T
): T {
	val transaction = NoSQL.createTransaction()
	val result = f(transaction)
	// Should not be under finally. If there's an exception, transaction is not committed
	transaction.commit().await()
	return result
}

So our transactions were not committed and we lost around 100M daily active users. As always, I totally blame Kotlin here and its language design.

How to protect against misplaced return in this situation? It’s very easy for developers to make such a mistake—seemingly the code is not different and developers aren’t careful because they are lulled by Kotlin and IDE that they can’t create bugs whatever they do… until shit hits the fan.

Why on earth does Kotlin require return for top-level functions anyway since it’s explicitly disallowed in lambdas?

Can we make return optional everywhere?

Sry to hear that you guys had a bug this bad. That said I don’t think making return optional is good. I personally would even like to go the other way and enforce return for lambdas instead of just returning the result of the last expressions. But neither change will happen since they are a breaking change without much or any real gain.

One fix that could help you prevent a bug like this to happen again is to move from inline functions to normal ones. Yes that would be a bit less efficent performance, but will also prevent non-local returns preventing this bug from happening again.
As a long term solution it’s maybe worth considering adding an annotation to kotlin that prevents non-local returns for inlined lambdas. Something like

suspend inline fun <T>withBatch(@NoNonLocalReturn f: (Transaction) -> T): T = TODO()
5 Likes

Have you considered removing the modifier inline?

Return is not explicitly disallowed in lambdas—it is allowed, it just does something different, which is sometimes (often!) what you want. The whole idea of nonlocal returns is to allow you to create constructions that work as if they were part of the language. You don’t expect an return inside of an if block to return only from the if do you?

The idea of constructs like this is that you can create things like…

fun doSomethingIfTasksAreDone() {
    tasks.forEach {
         if( !it.done ) return // Returns from the function
    }
    // All tasks are done...
    doSomething()
}

I suspect the problem is that you are coming from a language like Javascript and expecting Kotlin to behave the same way.

Respectfully, I’d like to suggest that if you have 100M DAU, you probably should be going through a testing and QA process before you deploy code into production. If you already have such a process, you might want to look into why something potentially this costly was able to get through it, and what could be done to improve the testing process (in the apps I work on, we have a critical test path that must pass even if we’re doing an emergency hotfix for, say, a security issue, to make sure that business-critical and revenue-critical parts of the app haven’t been affected by the change).

That said, of course, it’s nice to be able to catch as much as possible during development, before going to QA (and just in case QA misses it, before going to production). That’s where Kotlin and the IDE are nice but they are never a substitute for QA… it’s impossible for a language or IDE to prevent developers from making bugs (this is a mathematical fact; see Halting problem - Wikipedia). This is an annoying but unfortunate part of software development: As amazing as they are, tools can only take you so far.

There is a way to handle this better under the existing tools, though, so I won’t leave this without offering a suggestion :slight_smile:

My solution in cases like this is to use finally to commit the transaction. I see the comment about not using finally because you don’t want to commit the transaction if there’s an exception, but that’s easily remedied: Catch and rethrow any exceptions, and when doing so, set a flag somewhere to invalidate the transaction so it’s not committed on the way out via the finally block.

For example (I haven’t tested this, but I use a similar approach):

suspend inline fun <T> withBatch(
		f: (Transaction) -> T
): T {
    var validTransaction = true
	val transaction = NoSQL.createTransaction()
	
	try {
       val result = f(transaction)
       return result
    } catch (e: Throwable) {
       validTransaction = false
       throw e
    } finally {
       if(validTransaction) {
           transaction.commit().await()
       }
    }
}

Optionally, you could also have a flag like returnedNormally and set it to true after calling f(transaction), then check for it in the finally block and if it’s false, throw a fatal error to let the programmer know they returned the wrong way (if you want to disallow nonlocal returns). You could even have the fatal error be an assert so it only happens in debug builds…

(Please keep in mind the above code is untested)

I hope that was at least somewhat helpful (and I hope your business can recover from the loss).

10 Likes

Or add crossinline which is done for this specific purpose.

7 Likes

I totaly forgot about that. So I guess my suggested annotation already exists as a keyword :man_facepalming:

1 Like

Quick question: why does crossinline require a function to be marked suspend? Shouldn’t it inline the lambda, so it doesn’t matter?

It doesn’t. A function needs to be marked with inline so you can use crossinline, but it does not matter if it’s suspend or not.

1 Like

I have to write now:

suspend inline fun <T> withBatch(
		crossinline f: suspend (Transaction) -> T
): T {
	val transaction = NoSQL.createTransaction()
	val result = f(transaction)
	// Should not be under finally. If there's an exception, transaction is not committed
	transaction.commit().await()
	return result
}

Otherwise it will be a compiler-time error.

Your lambda is a marked with suspend so you can’t call it if your functions isn’t also a suspend function. Either You have

inline fun <T> withBatch(crossinline f: (Transaction) -> T): T {
    // ...
    f(transaction)
}

or you can add the suspend keyword to both the lambda and the function, but you can’t add it just to the lambda.

@Wasabi375 Without crossinline, suspend was not required on the lambda (see my initial code), with crossinline it is required… Please read a little more carefully the posts to which you are replying :slightly_smiling_face:

Thanks everyone for chiming in. I’m definitely walking away with a solution and additional knowledge.

I’d like to do one more push for my proposal for optional return. This is completely backwards compatible - return still works like before, but it is not required anymore.

I found that teaching return in Kotlin is really hard, especially to the people coming in from the world’s most popular language (JavaScript). I have to do it in three steps:

  1. return in top-level functions in Kotlin is just like in any other language.
  2. But, in lambdas there is no return but it will take the value of the last expression.
  3. However, if there is a return inside a lambda, then it will return to the top level function.

By 3 everyone is “What the heck?”…

Instead of this, we could just teach that in Kotlin return is not necessary and function will return the value of the last statement.

And then say that, if you really need it, you can still use return and explain all its peculiarities.

At the end of the day, return is just another unnecessary thing you have to type. And typing produces bugs. The less you type, the less bugs there are. Simple.

Which suspend keyword are you complaining about? The one on the lambda (which you have to add so the lambda can call suspending functions) or the one on your function required to call the lambda?

It’s realy not (maybe as long as you come from any other programming language). Return works just as in any other language. The only “new” concept is non-local returns(https://kotlinlang.org/docs/reference/inline-functions.html#non-local-returns). They are a super powerful feature enabling many features in the kotlin standard library. A simple way to think about return in kotlin is: return always returns from the last fun declaration (unless it’s a qualified return, eg. return@someQualifier). As described in the documentation this is just the same way return works from inside any other block (loop, if, etc). You don’t expect return to just return form a for-loop, you expect it to return from the function.

2 Likes

Glad to hear you have a solution!

I recently had the opposite problem going from Kotlin to JavaScript. I originally learned JavaScript before Kotlin but have mainly used Kotlin for a few years now, and recently had to start working in JavaScript again for a new project. The whole idea of “return” sometimes NOT returning from a function really threw me and led to several hours of frustrating debugging a couple of days ago (despite knowing how it works and having used it just two years ago). I get that lambdas are called “function” in JavaScript and that arrow notation is a shorthand for that, so return in a lambda is kind of return from a function, but… the “function” keyword is no longer there, so it’s still easy to forget and make mistakes, especially coming from Kotlin to JavaScript. Doing someArray.forEach(()=>{}) in JavaScript and having a nested return just terminate the forEach rather than the enclosing function seems somehow wrong.

The tools and environments surrounding modern JavaScript are truly amazing, but I do think that JavaScript syntax is somewhat weighed down by its history (just look at the “let” vs. “var” ugliness, and all the evil surrounding “undefined” vs. “null”). Kotlin had an opportunity for a fresh start and I’m glad they didn’t follow JavaScript or Java just for familiarity, even if that means it takes time to get used to the differences.

1 Like

@Wasabi375:

suspend inline fun <T> withTransaction(
		f: (Transaction) -> T
): T {
    // ...
}

// Compiles! No suspend on f required
fun main() {
    withTransaction {
        delay(1000)
    }
}
suspend inline fun <T> withTransaction(
		crossinline f: (Transaction) -> T
): T {
    // ...
}

// Does not compile!
fun main() {
    withTransaction {
        delay(1000)
    }
}
suspend inline fun <T> withTransaction(
		crossinline f: suspend (Transaction) -> T
): T {
    // ...

}

// Compiles again!
fun main() {
    withTransaction {
        delay(1000)
    }
}

It’s a minor point, however requiring suspend on f makes me suspicious it is really inlined.

Actually Kotlin uses as single straightforward rule: return always pertains to the innermost fun declaration. It allows you to forget the difference between a lambda and a built-in block.

4 Likes

Consider this code:

suspend fun callMe() {
    (1..10).forEach { suspendableFunction() };
}

forEach is not declared as a suspendable function and neither is its lambda parameter, but you are still allowed to call a suspendable function. This works because the code never becomes an actual lambda and it works more like a macro, putting all the code right there in the body of callMe() so there were no non-suspendable functions called on the path to suspendableFunction() invocation.

This is the reason your initial solution didn’t require a suspend on the lambda parameter. Once you request an actual, reified lambda, or even an inline one but that can be expanded into the body of another lambda, you must explicitly declare that you allow suspendable code.

1 Like

As always, I totally blame Kotlin here and its language design.

As always, I totally blame your tests here and your system design.
Kotlin can be improved, but you should always test. A good test would catch this kind of error “All transactions should commit”

7 Likes

So you’re saying crossinline is not actually inlined?

Thanks for catching my sarcasm.

I started championing this syntax in our codebase:

fun f(...) = run {
}

To me, this has a couple of benefits:

  • No need to specify return type (just like in TypeScript). This is a welcome convenience.
  • No need to specify return keyword, therefore avoiding a whole class of bugs.

My philosophy is, the less you type, the better.

I call this syntax @alamothe’s Smart Function Declaration Syntax (SFDS). You’re welcome!