Request: structured resource management kotlin library

Language Design
1

IMHO one of the limitations of Java is resource management: garbage collections works well enough when it comes to Java objects, but sometimes some native resources are referenced and are not garbage collected, or they are garbage collected only at a later stage but the developer wants to to promptly release then.
The java solution? Closeable, AutoCloseable.
Well, they work decently but they have some clear limitation:

  • They introduce an additional method to the interface.
  • Every object owning a Closeable resource must be Closeable itself if it doesn’t consume the resource. (for example, InputStreamReader is Closeable).
  • The resource can easily leak in different scenarios, for example during the transfer of the object reference to its owner:
class MyClass: Closeable {
    private val file1 = FileInputStream()
    private val file2 = FileInputStream() // If this fails then file1 is leaked

    override fun close() {
        file1.close()
        file2.close() // This is never executed if file1.close fails
    }    
}
  • One particular use case where they can easily leak without being notice is with coroutines. The following code looks safe, but it is not really. If the coroutine is cancelled during the withContext execution, the FileInputStream is leaked.
suspend fun fileInputStream() = withContext(Dispatchers.IO) {
    FileInputStream()
}

fun user() {
    fileInputStream().use {
        // do something
    }
}

This issue doesn’t have a real resolution: Universal API for use of closeable resources with coroutines · Issue #1191 · Kotlin/kotlinx.coroutines · GitHub

In general, Closeable works, but has some important limitations and makes the code prone to resource-leakage, bloated of close methods and with more compromises (a ByteArrayInputStream must implement close with NOOP).

The solution? It’s already present in the KEEP of context receivers: KEEP/context-receivers.md at master · Kotlin/KEEP · GitHub

interface AutoCloseScope {
    fun defer(closeBlock: () -> Unit)
}

context(AutoCloseScope)
fun File.open(): InputStream

fun withAutoClose(block: context(AutoCloseScope) () -> Unit) {
    val scope = AutoCloseScopeImpl() // Not shown here
    try {
        with(scope) { block() }
    } finally {
        scope.close()
    }   
}

// usage
withAutoClose {
    val input = File("input.txt").open()
    val config = File("config.txt").open()
    // Work
    // All files are closed at the end
}

Structured resources. Just like what coroutines did for concurrency, a new library can do for resource management.
This would move the resource registration from the interface of the object to the implementation:

interface MyInputStream {
    fun read(): Int
   // It doesn't have a close method or inherit from Closeable
}

class MyByteArrayInputStream: MyInputStream {
    override fun read()
    // No need for noop close
}

class MyInputStreamReader(
    private val inputStream: MyInputStream
) {
  // no close method, no Closeable interface
}

fun usage() {
    val reader = MyInputStreamReader(MyByteArrayInputStream())
    // no need to close anything :D 
}

context(AutoCloseScope)
class MyFileInputStream: MyInputStream {
    init {
        defer { closeMyFile() }
    }
}

fun usage2() {
    withAutoClose {
        val stream = MyFileInputStream()
    }
    // This is disallowed, it wont compile without an AutoCloseScope context
    // val stream = MyFileInputStream()
}

This solution solves many of the problems:

  • No additional close method is needed on the interface.
  • Objects that are not owning the resource don’t have to be Closeable (such as InputStreamReader)
  • It makes resource leaking much more difficult
context(AutoCloseScope)
class MyClass {
    private val file1 = MyFileInputStream()
    private val file2 = MyFileInputStream() // If this fails then file1 is NOT leaked
}
// release cannot leak as well
  • It works well with coroutines
context(AutoCloseScope)
suspend fun fileInputStream() = withContext(Dispatchers.IO) {
    MyFileInputStream()
}

// This is now safe and don't leak even during cancellation
suspend fun user() {
    withAutoClose {
        fileInputStream()
        // do stuff
    }
}

What is missing?
A more advanced solution that supports all the use cases, has a clear API and widely used.
Interoperability of different custom implementations of AutoCloseScope would be messy.
What I am suggesting with this post is an investigation around this possibilities and what it can potentially enable. Structured resource management could be a nice addition to the kotlin libraries and could tackle both the limitations of Closeable and the kotlin-specific problems related to coroutines.

3 Likes
2

Managing resources without leakage is a challenging issue. In particular when it involves other abstractions. It is easy enough to use/overload the use function, but it still allows iterators to be leaked. An AutoCloseScope can enhance this, but the way I’m currently leaning is to go the monad way. Meaning that the resource usage will be evaluated directly with its creation, therefore stopping any leakage (of course stale iterator are still possible if you’re being silly about things).