Await for Coroutine Completion during Shutdown Hook

Support
1

How can I gracefully shutdown a custom ExecutorCoroutineDispatcher, and await for all previously submitted coroutines to complete (within a given timeout period)? In my application there is work being dispatched to separate ExecutorCoroutineDispatchers for background execution to avoid suspending the calling function. I have observed that at application shutdown time, long running coroutines (on the order of seconds to 10’s of seconds) are not completing execution before my application is killed.

I originally tried implementing a shutdown hook which uses the idiomatic Java pattern of calling ExecutorService::shutdown and ExecutorService::awaitTermination on the underlying ExecutorService, but this does not work for the following reasons:

  • Calling shutdown prevents new tasks from being submitted (which I want)
  • Calling awaitTermination blocks while the currently executing tasks complete (which I want)
  • Coroutine suspension and resumption causes a new task to be submitted to the ExecutorService, which as previously mentioned is now prevented, meaning that my coroutines will never finish executing after attempting to continue from suspension

To illustrate this more clearly, consider the toy example below:

val executorService: ExecutorService = Executors.newFixedThreadPool(1) { runnable ->
  Thread(runnable).also {
    it.name = "Test-${it.id}"
    it.isDaemon = true
    it.priority = Thread.NORM_PRIORITY - 1
  }
}

val coroutineDispatcher: ExecutorCoroutineDispatcher = executorService.asCoroutineDispatcher()

CoroutineScope(testCoroutineDispatcher).launch {
  println("START")
  delay(100)
  println("END")
}

runBlocking { delay(10) } // Give some time for the coroutine to start

// This would be in the shutdown hook, omitted for clarity
executorService.shutdown()
executorService.awaitTermination(1000, java.util.concurrent.TimeUnit.MILLISECONDS) // Ample time for the coroutine to finish

// --- OUTPUTS: ---
// START

Please help me with a solution to block while I await for all previously submitted coroutines to finish (while preventing new ones from being started).

2

For anyone that stumbles upon this in the future, I ended up rolling my own solution as I could not find any well established pattern to achieve this.

The idea is to use a long-running parent job, whose only responsibility is to launch child coroutines from within itself. Then at shutdown I can await and block on the parent job to complete, and Kotlin’s structured concurrency ensures this will only happen when all children have finished. In order to launch jobs, I use a Channel to pass messages into the long-running parent job.