Kotlin needs try-with-resources

Here are the problems with the Closeable.use extension method:   * Doesn't work with Autocloseable (easy to fix, but seriously? this is a can of worms, but Kotlin's stdlib needs to be 100% compatible with whatever the latest JDK is. In fact, you probably need to version the stdlib (and, perhaps, Kotlin itself) together with the JDK.)   * Awkward to handle multiple resources   * Often needs a try block anyway to catch the exceptions coming from close(), the constructor, or the body   * Uses ad-hoc syntax

What do I mean by ad-hoc syntax?

It always has to be used like:

Files.newOutputStream(path).use {
  it.write(‘a’)
}


It shouldn’t be used like:

val ostream = Files.newOutputStream(path)

ostream.use {
  // by it value you can get your OutputStreamWriter
  it.write(‘a’)
}
… Oops, we’ve closed our ostream but the variable is still in scope, waiting to throw an IOException

Using try without try-with-resources is even uglier in Kotlin than it was in Java, because you must declare the variables as nullable and mutable.
var ostream : OutputStream? = null
var ostream2 : OutputStream? = null
try {
  ostream = Files.newOutputStream(path)
  
ostream2 = Files.newOutputStream(path2)
} catch (val e : Exception) {
  if (ostream != null) try { ostream.close() } catch (t : Throwable) { e.addSuppressed (t) }
  – Does Kotlin automatically suppress exceptions?
  – Regardless, we need to do it explicitly else we’d need 2 levels of try/catch
  if (ostream2 != null) try { ostream2.close() } catch (t : Throwable) { e.addSuppressed (t) }
  throw e

}

4 Likes

Absolutely agree. But we want to avoid language construct just for this as much as possible and we have an idea of the solution, it's not implemented in stdlib yet, but you can try it using code like this:

 
class ResourceHolder : AutoCloseable {
    val resources = arrayListOf<AutoCloseable>()

  fun <T : AutoCloseable> T.autoClose(): T {
  resources.add(this)
  return this
  }

  override fun close() {
  resources.reverse().forEach { it.close() }
  }
}

 

fun <R> using(block: ResourceHolder.() -> R): R {
  val holder = ResourceHolder()
  try {
  return holder.block()
  } finally {
  holder.close()
  }
}


 
fun
copy(from: Path, to: Path) {
using {
val input = Files.newInputStream(from).autoClose()
val output = Files.newOutputStream(to).autoClose()
input.copyTo(output)
}
}


Of course, it’s not production ready code, it’s just to demonstrate an idea. E.g. "close() in holder’s code should handle exceptions, etc.

5 Likes

Wow, that's a great idea that even beats most built in constructs that adresses this kind of issues. Your idea even allows for conditional creation of closable resources which is much harder with most common using/try-with-resources constructs, e.g.:

using {   val resource = if (isLocalhost()) {   ClosableResourceA().autoClose()   } else {   ClosableResourceB().autoClose()   }   ...

}

1 Like

I must admit, it's pretty cool.

But to provide some constructive criticism:
  * This code is not 100% safe. In particular, adding to a list is not safe from exceptions (OutOfMemoryError), so you’ll get into a situation where you lose track of a resource and don’t close it. If you use a fixed-length array, you’ll run out of room.
  * You’ll still often need a try/catch block
  * This syntax is still a little ad-hoc (much better, yes, but you still have to remember to add “.autoClose()”)

Why do you say “we want to avoid language construct just for this as much as possible”?

The point of language constructs (at least the really good language constructs!) is to cement a “pattern” from being some ad-hoc boilerplate code into a syntax that jumps out at the user and tells them to “use it correctly!” Gotos became if/for/while, structs with function pointers became objects, continuations became exceptions and async/await, etc, etc. Yet, there’s not many code patterns more fitting of a language construct than correct resource lifetime management. Java and C# both fucked up in not even shipping Closeable/IDisposable in versions 1. (The hubris!) Then they fucked up in how they implemented Closeable/IDisposable. (Java had to release another class, AutoCloseable, and C# requires developers to read giant articles like http://www.codeproject.com/Articles/15360/Implementing-IDisposable-and-the-Dispose-Pattern-P to even write a class that implements IDisposeable. Microsoft actually “fixed” it again when it created managed Visual C++ by generating the boilerplate automatically https://msdn.microsoft.com/en-us/library/ms177197%28v=vs.100%29.aspx , which is one of the very few advantages of Visual C++ as a managed language. Yes, the irony.)

Don’t let this happen to Kotlin! As this author http://www.codeproject.com/Articles/29534/IDisposable-What-Your-Mother-Never-Told-You-About pointedly states, “Never design a language with exceptions that do not also have deterministic resource deallocation.”

So, I would advise you to consider this subject seriously, and support the user with solid language constructs to deal with resource lifetime management. Even if we just make your “using” function into a keyword, we’re cementing it with syntax highlighting, and prohibing people overriding, overloading, or otherwise messing with the semantics. I’m sure there’s plenty of other things that could be tidied up along the way. I’m afraid I don’t have a ready list, but a pain point that comes to mind is implementing close for a class that has multiple autocloseable fields (which it owns):

class MyClass : AutoCloseable
{
  Closeable a = new MyCloseable();
  Closeable b = new MyCloseable();
  Set<Closeable> c = …;


  @Override public void close() throws Exception // Really icky. But we can’t just throw IOException, since the type of ‘t’ is unrestricted.
  {
  Throwable t = null;
  try { if (a != null) a.close() } catch (Throwable t2) { if (t==null) t = t2; else t.addSuppressed (t2); } // We want to catch Errors as well, since there’s no reason to not try  to dispose of our resources if there’s been an Error (the Error may be  caught and the program may continue).
  try { if (b != null) b.close() } catch (Throwable t2) { if (t==null) t = t2; else t.addSuppressed (t2); } // A trickier case is the InterruptedException, but I think we should catch those as well (since interrupting a thread is not guaranteed to work)
  for (Closeable cc : c) try { if (c != null) c.close() } catch (Throwable t2) { if (t==null) t = t2;  else t.addSuppressed (t2); } // Oh, geez, do we need to put for inside a try block as well??
  if (t != null)
          throw t; // I wonder, if this exception also gets suppressed, will the suppressed exceptions in this exception still get printed? Or will we need to “unroll” them?
  }
}

Maybe we can have ‘autoclosed var’ and ‘autoclosed val’ which will close the object whenever it leaves the scope? No “using” keyword required. For fields/properties, ‘autoclosed’ generates a ‘close’ method. This should be enough for most AutoCloseable classes (which just need to close their fields). For classes with more complex close logic, the user can specify one or more “destructors” which will be called from the generated close method using the appropriate try { } catch { suppress } logic. Additional features would be correct chaining of ‘close’ methods in the inheritance heirarchy, calling any finalizer logic from ‘close’ (and making any class which has a finalizer AutoCloseable), checking if the object has been closed on every method call (or at least maintain an isClosed property which can be manually checked), not running the destructors/finalizers more than once, etc. Maybe even a ‘close’ keyword for use inside a destructor (to catch the exceptions, as above). Basically, take a look at what Visual C++ does on the CLR.

closeable class MyClass
{
  autoclosed val a = MyCloseable();
  autoclosed val b = MyCloseable();
  var c : Set<Closeable> = …;


  destructor ()
  {
  for (val cc in c)
  close cc; // Gotta close 'em all!
  }
}

To go the final mile, we can also re-implement finalization according to this advice http://www.oracle.com/technetwork/java/javamail/finalization-137655.html in order to mimic CLR’s GC.SuppressFinalize to avoid the finalization performance penalty http://lukequinane.blogspot.com/2009/03/java-suppress-finalizer.html if close has already released all resources.

The article I linked to previously (http://www.codeproject.com/Articles/29534/IDisposable-What-Your-Mother-Never-Told-You-About) highlights other big-picture issues with the dispose pattern, such as having to pre-determine object ownership and how it really fucks with the Liskov Substitution Principle in that if you have an object which implements interface Foo which does not derive from AutoCloseable, you don’t actually know whether the object itself implements AutoCloseable and needs to be closed (!!!). However, the proposed solution, reference counting, seems too radical for Kotlin. The article also brings up a laundry list of specific issues in the CLR, but I haven’t parsed it to figure out which apply to the JVM. (Generally, the JVM has much fewer such issues.)


Btw, you know what would be a quick fix to point #2 and a great addition to the language in general? Allowing catch/finally blocks to follow any block, permitting code like:

using {
  val input = Files.newInputStream(from).autoClose()
 val output = Files.newOutputStream(to).autoClose()
  input.copyTo(output)
}
catch (IOException e) {
  ;

}

as well as:

fun foo() {
  …
}
catch (…) {
  // look ma! no nested scopes!
}
finally {
  // let’s test postconditions
}

“try” is no longer a keyword

2 Likes

Yes, it's nice. The use of an extension function may cause unexpected issues though. For instance if you refactor some code out of the using block into a separate method, suddenly the autoClose method disappears, doesn't it? If autoClose was always present and would push the resource onto a Cocoa style autorelease pool, it'd fix that issue, but  at the cost of less type safety.

It doesn't disappear, if extracted function will have ResourceHolder as a reciever, which indicates that function requires external resource management. It actually allows for some nice composition techniques, and if ResourceHolder is interface, even passing resource cleaning strategy, etc.

1 Like

Bump.

This is important.

2 Likes

What about:

package info.macias.kotlin

inline fun <T:AutoCloseable,R> trywr(closeable: T, block: (T) -> R): R {
    try {
        return block(closeable);
    } finally {
        closeable.close()
    }
}

Then you can use it the next way:

    fun countEvents(sc: EventSearchCriteria?): Long {
        return trywr(connection.prepareStatement("SELECT COUNT(*) FROM event")) {
            var rs = it.executeQuery()
            rs.next()
            rs.getLong(1)
        }
    }

How is that better than use?

It appears that is use, but based on AutoCloseable instead of Closeable. Many things that implement AutoCloseable also implement Closeable, but some do not. However the issue is that AutoCloseable is not available in Java 6, but Closeable is and Kotlin targets only Java 6.

It occurs to me that there could be a use for a more general purpose use:

inline fun <T,R> use(t: T,  onExit:T.() -> Unit, block: (T) -> R): R {
    try {
        return block(t);
    } finally {
        t.onExit()
    }
}

The general purpose function with an onExit is very useful. That does not mean that an autoclose function would not be very useful as well (extending on your base, in this case with Closeable, but overloads to other types would be acceptable, for example to the jdbc types that for some reason don’t implement closeable):

inline fun <T:Closeable, R> autoClose(t:T, block: (T) -> R) = use(t, { close() }, block)

What would be really cool is if the following code would actually compile (maybe using some sort of template like SFINAE approach), note that this fails if T does not have a close method, but that the method could be resolved at compile time:

inline fun <reified T, R> autoClose(t:T, block: (T) -> R) = use(t, { close() }, block)

I am aware however that the error messages that can generated if this is brought forward can be horrific (see C++ template substitution failure)

It is not better itself. The only fact is that “use” is only working for “Closeable” classes, not for “AutoCloseable”.

And the reason is that Kotlin only targets Java 6 and AutoClosable did not exist in Java 6. If you are only targeting later Java it is trivial to add to your code

For those who do not want to implement the utility themselves and while Kotlin developers think whether to add it or not to stdlib, you may use a basic implementation (as described above) already available in Maven Central

<dependency>
    <groupId>net.offecka</groupId>
    <artifactId>kutils</artifactId>
    <version>0.0.1</version>
</dependency>

Targeted JRE 8, ancient stuff is not going to be supported…
Ever!

I completely agree with the point that new language construct should be avoided. But the lack of the try-with-resources (a feature that both Java and C# have) is a step backwards. It’s a fundamental concept which is a must-have in language without destructors. It deserves a proper syntax and a language construct which would make it fool-proof, as safe as possible and autocomplete-, fixit- and diagnostcs- friendly. It’s 10 times as important as any other candidate for a language construct.

@cubuspl42 That’s right, but given that all of that could be achieved with the help of standard use function, is there still a reason to introduce special language construct for that?

I completely agree with @alexd457. I took the idea presented here and extended it a little further, please let me know what you think.

Here is the implementation.

And usage examples via unit tests.

But seriously, none of this should be remotely necessary. This is something that should absolutely be a language construct. It makes sense for it to be, and would solve a whole host of problems.

This + overload is the exactly reason why many people don’t like operator overloading. + is about numbers, not about resource management. I would suggest just to use some function name. Otherwise great idea, I didn’t know that you could “overload” catch like that. I made similar library for myself. Honestly Jetbrains should just add something like that to standard library along with use.

I agree about + overloading. Better to use something like use or even resource to avoid name clashes. As for the whole discussion, I think it is a great idea to move a lot of features from language into library, it simplifies understanding of the code and gives a lot of additional freedom.

I also think that try-with-resources should be in library functions.
On the other hand, I think that a seperate project “Kotlin-EE” like “Java-EE” should be maintained, where things like this should be standardized.
But maybe it’s to early for that, yet?

1 Like