In future, could Kotlin have checked exceptions?

[acarlstein]

John (a fictional character in this story), just died.
The machine that was keeping him alive stop working properly.

The machine was controlled by an Android device.
Even do the app was wrote in Kotlin, most of the libraries related with Android were wrote in Java.

One of the nurses, just happen to try to send a report by sharing it with another app.
In that moment, the app crashed.

IT Forensics took a look into the log and found out this exception: PackageManager.NameNotFoundException

When the developer was approached, they found out that the person was new in Kotlin but had a background in Java.

Normally, when working with Java, modern IDEs would provide some warning when an exception needed to be caught:

Please notice the red line below the method sleep, indicating to the developer that there is a problem that should be taken care of

This feature allowed the developer to cover most of the cases of things that could go wrong. Just leaving weird cases to be discover later.

However, when working with Kotlin, the IDE was quiet:

Yes. The developer could hover the mouse over the method and wait until a box would show up and read that there was an exception; however…

Why the IDE was quiet then?

Why the IDE didn’t mark the method to make it easy to spot a possible problem?

Could it be that those behind the IDE forgot to provide some mechanism to detect these exceptions and report them back to the developer as when developing with Java?

Could it be that when developing the IDE that would parsing the Kotlin code, they didn’t add the feature due Kotlin’s documentation? Exceptions | Kotlin Documentation

I mean, Kotlin has a different approach as shown in the video below:

KotlinConf 2019: Error Handling Strategies for Kotlin Programs by Nat Pryce & Duncan McGregor

Yes. It is the fault of the developer because didn’t take the time to go over the documentation of every little thing he or she encounter and trust the IDE; however, isn’t the idea of using a smart IDE to speed up development?

Or perhaps is that Java still remain behind the scenes. Java has different rules than Kotlin and Kotlin should support them, if we insist that Java should remind behind the scenes.

Perhaps, Kotlin could actually change the paradigm related with Exceptions if would get rig of everything related with Java behind scenes?

However, until that day arrives, is it possible to provide something that could help the IDE’s to notify the developers about these exceptions in an active way? How about an Annotation? Something.

Anything that could help to prevent to go over every little piece and check if there are some exceptions to take care of?

Or this is something that IDE should take care of?

[elizarov]

We do have a plan for effects and coeffects via the multiple receivers. They, indeed, help with organizing error-handling strategies in code. You can read more in the corresponding KEEP - KEEP/context-receivers.md at context-receivers · Kotlin/KEEP · GitHub

[pdvrieze]

For an application like the one you describe it should be something that a static analyzer is used to detect and flag. It is a cheap inspection that could also be integrated into the IDE default analysis.
Unfortunately exceptions are a deceptively attractive, but in many ways broken, in particular in the various contexts where exceptions pass through a library layer. Neither exceptions wrapping exceptions; excessively long expected exception lists (many of which cannot be handled by the caller anyway); nor overly generic exceptions (eg. throws Exception) are desirable. A better approach is not definite, but things like Result monads seem more promising. Most importantly, there is nothing stopping you from having a coding standard (enforced by static analysis) that forces catching, but you will have to deal with the fact that many Kotlin libraries do not declare thrown exceptions.

As to correctness, one of the challenges here is that exceptions are meant for stuff that is never supposed to happen, not just to support invalid inputs made by a user (entering text into a field that allows that but expects numbers is not an error from the program perspective). Also keep in mind that writing exception safe code is extremely hard, and in your life-and-death case, you would need to build tests to ensure that your code is exception safe (if an exception is thrown anywhere the system is always in a valid - and correct - state).

[HughG]

Speaking as someone who writes medical software for a living, although I would love to see better support around error handling in all sorts of languages (a) adding some IDE feature to warn you about Java checked exceptions propagating into Kotlin still wouldn’t handle RuntimeException being thrown where it shouldn’t be or wasn’t expected, from one of your many Java dependencies; and (b) if your quality system relies so heavily on IDE warnings to prevent loss of life, you ought to be heavily fined, or never certified in the first place!

Also, showing warnings for Java checked exceptions in a Kotlin context might lead the developer to gain a false sense of security, since exceptions from Kotlin code won’t be highlighted. It’s not the developer’s fault, necessarily, but it is the company’s fault for allowing them to work on safety-critical software without adequate training.

[acarlstein]

Hugh,

My previous message was a general example.

If you write medical software, then you should know that most of the time, C is the language used.
The reason is that C allows the developer to be closer to the hardware that must control.
A good C developer can have a good clue on the Assembly being created.

How do I know? Well, my former roommate quit the field after he couldn’t handle having to install electrodes inside monkeys brains.
They would analyze the signals when the monkey would move its eyes and other things.

Regardless, we are not here to talk about your credentials or mine, Hugh.
We are here to talk about Kotlin.

Now, don’t get me wrong. I like Kotlin. I think is great.
However, the issue here is that Kotlin is working together with Java and Java has some idiosyncrasies that Kotlin isn’t addressing.

You see, as a developer, I’m counting in the documentation, the API, and the libraries wrote in Kotlin to do my work.
To provide another example, if I’m building a parser, an analyzer, or some short of auto-help, for example, I am going to use those things.
If those things are missing the idiosyncrasies from Java, when Java does produce a side effect, then we have a problem at hand.

Also, there is a matter of speed Hugh.
Why do developers must be breaking into proprietary code or spend hours going over every line of code in relation with Kotlin to do their job?
There is a reason we create tools and solutions to write robust code faster.

Plus, the IDE is only one of those tools. You know that there are unit test, automated test, solutions such as SonarCloud to verify code smell and short.
However, the fact is that the documentation is leaving side effects, coming from Java, out. That’s a big no.

So, either lets take Java out of the equation or lets build something that takes the idiosyncrasies of Java in account.

[taylan.kammer]

Following are my two cents as someone who’s now been developing on Android for about 5 years, and a bit of server-side Java here and there. This is a long discussion and I haven’t carefully read through all of it so I don’t know if I’m contributing anything new below…

I’ve come to love and embrace checked exceptions. Although I’ve had some headaches along the road with my heavy functional-programming style(*), I’ve discovered a while ago that you can use type parameters for the “throws” clause as well, solving the severe issue with things like the “visitor pattern” where you deal with unknown exception types.

(*) For instance I have a utility class Fn for typical higher-order functions like map, fold, filter, find, etc. which I use very often, much nicer than Java’s overly verbose “streams” API. I also make heavy use of higher-order functions in some internal APIs in my codebase.

Normally you’re forced to turn everything into a RuntimeException when making use of higher-order functions a la “visitor pattern,” but type parameters in throws solves that issue.

Here’s my implementation of map which turns a List<T> into a List<S> via a Mapper<T, S, U>, which is like a Function<T, S> except it may throw an Exception of type U which is cleanly propagated upwards to the map.

public interface Mapper<T, S, U extends Exception> {
    S apply(T input) throws U;
}

public static final <T, S, U extends Exception> List<S> map(Collection<T> inputs, Mapper<T, S, U> mapper) throws U {
    List<S> outputs = new ArrayList<>(inputs.size());
    for (T input : inputs) {
        outputs.add(mapper.apply(input));
    }
    return outputs;
}

It does exactly what it should. I could handle the checked exceptions thrown by the Mapper inside of its own body, like forcing it into an unchecked exception:

// This is just an example, there are better ways to do this!
private List<String> getQueryParams(String uri, String encoding) {
    List<String> encodedQueryParams = getRawQueryParams(uri);
    List<String> queryParams = Fn.map(encodedQueryParams, qp -> {
        try {
            return URLDecoder.decode(qp, encoding);
        } catch (UnsupportedEncodingException e) {
            throw new RuntimeException(e);
        }
    });
}

Or! I could let it propagate to the higher level:

private List<String> getQueryParams(String uri, String encoding) throws UnsupportedEncodingException {
    List<String> encodedQueryParams = getRawQueryParams(uri);
    List<String> queryParams = Fn.map(encodedQueryParams, URLDecoder::decode);
}

Neat!

The only annoyance is that it makes the implementation of higher-order functions more verbose, but the payoff is definitely worth it.

In light of this, I see no convincing argument against checked exceptions apart from subjective taste (which may be based on years of experience, but still). Of course, adding them to the language at such a late stage would cause unacceptable compatibility issues, but I wonder whether it could be offered as an optional feature.

[swapnilg]

@pdvrieze With Kotlin Coroutines coming into picture and having a capability to attach an exception handler on your coroutine scope. I find that even more powerful than having a try-catch around the checked exception. We have a single sink for all my exceptions which gives a lot of power to the edveloper.

[negora]

In my opinion, the concept of checked exceptions is very useful. The problem is the tools that we’re given to work with them.

Because of this, weeks ago, I made 2 proposals in one of the Java mailing lists. Unfortunately, they didn’t had any success. So I publish them here (with some modifications) with the hope that they gain some support for Kotlin.

Exception inference.

It has already been proposed here. To me, it’s a essential feature so that checked exceptions are bearable to both those who like them (me) and those who hate them.

In my original writing for Java I proposed to limit it to private methods. The reason is that I consider type inference to be risky in non-private boundaries. Because you can accidentally change the signature of a method very easily.

In addition to that, I also proposed that exception inference should be requested explicitly with throws * or throws uncaught, which basically means throw all those checked exceptions that haven’t been caught in a try-catch block. Obviously, this makes that exceptions part of the method’s signature.

Demotable exceptions.

Behind this term I propose the ability to make checked exceptions behave as unchecked ones forever or just temporarily, depending on the need.

How? With an operator prepended to the exception name in the throws clause. I thought on ~ as the operator.

When do we need to demote an exception forever?

Suppose a method that loads a Student from a repository. Internally, the class uses a database to store data but we want to hide that fact from the user. So we would do this:

fun loadStudent (id: Int): Student
		throws StudentNotFoundException,
			~DatabaseException,
			~IOException {
	// Code that throws 3 checked exceptions.
}

The ~ operator would make DatabaseException and IOException behave as unchecked exceptions. And, in addition, it would exclude that exceptions from the signature of the method.

Thanks to this feature, try-catch blocks would be used only when we need to do something more interesting than wrapping a checked exception to hide it from the outer world. And we do that a lot in Java.

When do we need to demote an exception temporarily?

Suppose a method that expects a Stream of Students to store them somewhere:

fun storeStudents (students: Stream<Student>): Void {
// Code...
}

But in the caller, instead we have a Stream of lines (Strings) that first need to be parsed and converted to Students. And such operation may throw a ParseException. We would do something like this:

fun parseAndStoreStudents (lines: Stream<String>): Void {
	storeStudents (lines.map (
		{ line -> parseStudent (line) }))
}

fun parseStudent (line: String): Student
		throws ~ParseException {
		// ^^^ Here we're demoting the checked exception to unchecked.
	// Code...
}

This would work, but the problem is parseAndStoreStudents() should propagate ParseException as checked, so that the caller is aware of that possible error. To do so, we would just promote that exception as checked again, by adding it to the throws clause without ~:

fun parseAndStoreStudents (lines: Stream<String>): Void
		throws ParseException {
		// ^^^ Here we're promoting the exception back to checked.

	storeStudents (lines.map (
		{ line -> parseStudent (line) }))

}

fun parseStudent (line: String): Student
		throws ~ParseException {
	// Code...
}

What about lambdas?

In this case we could use a variation of the -> operator, such as ~>. Example: { a, b ~> a + b}. Or maybe we could prepend the operator to the whole lambda. Example: ~{ a, b -> a + b }. Anyway, the operator would apply to all checked exceptions thrown in the body of that lambda, as long as they have not been caught in a try-catch block.

The previous example would look like this:

fun parseAndStoreStudents (lines: Stream<String>): Void
		throws ParseException {

	storeStudents (lines.map (
		~{ line -> parseStudent (line) }))

}

Conclusion.

In my opinion, these 2 features would make work with checked exceptions as pleasant as if you used unchecked exceptions only, but making the code much safer.

PD: Sorry if there are syntax errors in the examples. I don’t use Kotlin (yet) precisely because of the lack of checked exceptions or a better mechanism.

[owen.paul.thomas]

Here’s a quick reply to this post. I have been developing in Java SE for about 14 years, and although I think checked exceptions are a good thing, I think their implementation does have a few drawbacks.

I think Java’s use of the base Exception class as the checked type was wrong, so perhaps Kotlin could use an abstract exception, call it perhaps CheckedException, where this, and subtypes can be checked at compile time.

Additionally, in one’s deployment descriptor for one’s module, one can indicate that CheckedExceptions of specific subtypes are thrown, caught, or ignored. If thrown, any dependent modules must indicate that the appropriate type is caught or ignored, if caught, one’s IDE indicates when an exception that is thrown is not being caught, and if ignored, no checking is performed.

In the final case, the compiler keeps track of which functions implicitly throw a CheckedException or subtype so that any calls to these functions from dependent modules must be handled if these dependent modules indicate that the corresponding exception type is caught.

This might be a bit of work, but I think this is theoretically possible. Have a great day.

[arkanovicz]

Checked exceptions don’t cope very well with closures. From my experience with Java, I found it a real nightmare.

How do you propose to handle the simple use case of a closure (anywhere in a method expecting a closure in the standard library, for instance) calling a piece of code throwing a checked exception? In java, you either have to redefine functional interfaces and their call site methods, or to put try/catch clauses everywhere.

[owen.paul.thomas]

Hmmm… I haven’t written any closures in Java that weren’t over my own methods, and being that my method signatures can be changed by me, I haven’t had to deal with this case. The obvious thing that I would do here is wrap my checked exception in a RunTimeException and hope that I can catch it when this exception falls out somewhere else in my code.

Perhaps the “thrown” clause can be implied (or stated) in some way for closures where a case like this happens so that if your module closes over a method from any dependent modules, those methods must also include appropriate caught or ignored clauses in the deployment descriptor? Other code that depends on your module might then be able deal with the unwrapped checked exception appropriately at compile time?

I haven’t had much to do with Lambdas in Java, but perhaps this might work here as well?

HTH.

[broot]

@arkanovicz

How do you propose to handle the simple use case of a closure (anywhere in a method expecting a closure in the standard library, for instance) calling a piece of code throwing a checked exception?

Well, this shouldn’t be a problem. We have exactly the same case with suspend functions and also with non-local returns. For example, we don’t have to duplicate list.map() for suspendable and non-suspendable variants, because Kotlin can “pass through” higher-order functions if they are fully inlined.

[mtimmerm]

@owen.paul.thomas

I think Java’s use of the base Exception class as the checked type was wrong, so perhaps Kotlin could use an abstract exception, call it perhaps CheckedException, where this, and subtypes can be checked at compile time.

It was a mistake to make the checked vs unchecked determination based on the type of the exception. I think just about all the problems with checked exceptions would go away if we used a rule instead, that if you call a method with an exception in its throws clause, then that exception must be explicitly dealt with.

[arkanovicz]

You tell it yourself: “if they are fully inlined”. But there are some limits to inlining. Everything cannot be inlined. And it’s not even desirable…

[broot]

@mtimmerm

I think just about all the problems with checked exceptions would go away if we used a rule instead, that if you call a method with an exception in its throws clause, then that exception must be explicitly dealt with.

Frankly, I don’t see how this would solve anything. Exceptions propagation is one of their main features. People would just catch exceptions and immediately rethrow them - additional code, degraded performance.

@arkanovicz

You tell it yourself: “if they are fully inlined”. But there are some limits to inlining. Everything cannot be inlined. And it’s not even desirable…

Of course there are limits to inlining. But still, I think this problem sounds more serious than it is in practice. You even mentioned the standard library as an example. Well, stdlib doesn’t have any problems with suspendable functions, so it wouldn’t have with checked exceptions as well. This is because almost all higher-order functions in stdlib that call lambdas directly, are fully inlined.

[owen.paul.thomas]

@mtimmerm

It was a mistake to make the checked vs unchecked determination based on the type of the exception. I think just about all the problems with checked exceptions would go away if we used a rule instead, that if you call a method with an exception in its throws clause, then that exception must be explicitly dealt with.

While I think your argument has some resonance, I also think it is good to have a specific checked exception type because the programmer can scope a set of deleterious events that they have anticipated. I think perhaps a CheckedException class would be a good starting point from which a library of such events can be built for one’s deployed module, and any users of one’s module could easily navigate their way through one’s library of anticipated exceptions if they wanted to.

The main gist of my suggestion was in finding an acceptable solution to situations where a developer didn’t want to deal with a particular problematic event that the developer of a dependent module had anticipated.

Likewise, if one creates a method that one intends dependent modules to call, I think it would be good if the developers of those dependent modules could control whether they intend to deal with the exception, or whether they prefer to ignore it and pass this responsibility onto modules that are dependent on theirs in turn.

In my suggestion, if other developers are able to ignore an exception from one’s library, there would be less likelihood of these other developers burying checked exceptions within unchecked ones because at some point, every checked exception might be dealt with appropriately; not every exception would need to be dealt with by the next dependent module, and so the necessity of dealing with any particular checked exception can be deferred to an appropriate handler module and the developers of modules between these can get on with their lives.

I think this type of flexibility would generally be appreciated and eventually adopted by Kotlin developers if it were available. Kotlin might also recruit those Java deveopers that are waiting for a checked exception mechanism in Kotlin.

[sylvainspinelli]

Note: an alternative of checked exception with (future) union types : https://youtrack.jetbrains.com/issue/KT-13108/Denotable-union-and-intersection-types#focus=Comments-27-6024529.0-0

fun foo(): Int | MyException1 | MyException2 {
    if (/*...*/) return 1
    return MyException1()
}

when (foo()) {
    is Int -> TODO()
    is MyException1-> TODO()
    is MyException2-> TODO()
}

[negora]

I’m sorry for repeating myself but, in my opinion, the concepts that I mentioned in a previous message are necessary so that checked exceptions have some acceptance in a language like Kotlin:

1. Have checked exception inference in private methods, so that it’s not necessary to write a throws clause every time. Non-private methods would act as barriers in which the developer would have to stop and think which exceptions want to be exposed in the public API.

2. Be able to demote (even temporarily) a checked exception into an unchecked one, by means of putting a prefix before its name, in the throws clause. No need to use a try-catch block and wrap the exception with an unchecked one just to make it traverse the current layer.

With regard to inlining lambdas and make checked exceptions “transparent”, that may work if the lambda is used and then discarded in the current function or method. But if it escapes from that scope, then it won’t work. In Java, for example, it often happens when you return a Stream and have applied a map() and/or filter() operation prior returning it. The lambdas are not executed immediately, but stored somewhere, waiting for when the terminal operation is called. So the exceptions can’t be (and mustn’t be) thrown transparently by the method where lambdas are declared.