Detached coroutines

Libraries
1

It is a continuation of previous discussion here, but I decided to open new topic.

The question is about the best way to use lazy detached coroutines. I mean the case,when coroutine is created (without starting) in one place and then called in another place. I use it to create lazy task graphs, but it would be useful for other future usages like serializable coroutines and distributed computing.

Current problem is that if I create a lazy Deferred in a scope and then call it outside of it, it won’t work since parent scope is closed. Basically, detached coroutines violate structured concurrency. There are two solutions:

  1. Use GlobalScope or some other scope that is guaranteed to span over both creation site and use site. It seems not the best solution, since we completely miss all structured concurrency benefits. Also, if we consider serialized coroutines, it won’t help since they are produced outside of even large scope.
  2. We can bundle a suspendend funcition with coroutine context and transform it to actual Deferred on call site using provided scope. It seems like a solution, but requires some additional work.

Any thoughts about it?

2

Yes, it could.
But I think that each of these problems raises different issues and requires different solutions: distributed structured concurrency is not a direct consequence of the local one.

I read: you should use a larger scope that used.

I consider this the best solution if the Deferred scope is the application lifetime.

Global scope is used to launch top-level coroutines which are operating on the whole application lifetime and are not cancelled prematurely.
https://kotlin.github.io/kotlinx.coroutines/kotlinx-coroutines-core/kotlinx.coroutines/-global-scope/

I don’t know how your application work, I assume now for example that your computation is valid inside a HTTP session.
When a HTTP starts you can attach a scope to it, so all tasks and dependencies live in it. Dependencies will executed once, tasks can be executed on each HTTP request.
HTTP scope is not the GlobalScope, its lifetime is shorter and this scope is cancelled when the HTTP session is invalidated.

This same solution is used to handle scope lifetime in Android’s activities.

So, in my opinion, GlobalScope is a valid solution, but not the only one.

3

I’ve ended up doing something very similar (yet more elegant) to what I’ve done with CompletableFuture back in Java:

open class DynamicGoal<T>(
    val coroutineContext: CoroutineContext = EmptyCoroutineContext,
    override val dependencies: Collection<Goal<*>> = emptyList(),
    val block: suspend CoroutineScope.() -> T
) : Goal<T> {

    final override var result: Deferred<T>? = null
        private set

    /**
     * Get ongoing computation or start a new one.
     * Does not guarantee thread safety. In case of multi-thread access, could create orphan computations.
     */
    override fun startAsync(scope: CoroutineScope): Deferred<T> {
        val startedDependencies = this.dependencies.map { goal ->
            goal.startAsync(scope)
        }
        return result ?: scope.async(coroutineContext + CoroutineMonitor() + Dependencies(startedDependencies)) {
            startedDependencies.forEach { deferred ->
                deferred.invokeOnCompletion { error ->
                    if (error != null) cancel(CancellationException("Dependency $deferred failed with error: ${error.message}"))
                }
            }
            block()
        }.also { result = it }
    }

    /**
     * Reset the computation
     */
    override fun reset() {
        result?.cancel()
        result = null
    }
}

The computation actualizes only when start is called, not when it is created. Then it propagates the scope back to its dependencies. If some of dependencies are already started, then they use the scope they are assigned (we assume that computation result does not depend on the scope). I did not look into it, but it seems that Flow is doing something similar, when it propagates execution context backwards.