As I understand it, by default, if you start a Kotlin Coroutine via launch or async it’ll launch in CommonPool (or if you use GlobalScope ). And CommonPool is a ForkJoinPool and that, by default, is in non-async mode so it executes tasks in LIFO order. That seems like a poor choice for something like asynchronous web server applications where we’d want fair scheduling: we don’t want the poor sucker who hit our web server first to wait for all calls that came later. I did run some stuff in a debugger and it seems that the co-routine scheduling is done by calling execute on the ExecutorService so I think there isn’t any Kotline stdlib code there to help out.
I know that if there are free threads in a ForkJoinPool they’ll work steal and that can help, but I’d imagine that in a web server experiencing a burst of traffic that’s keeping all threads busy the LIFO nature of CommonPool would mean that metrics like 95th percentile latency would be worse (average latency might actually be better due to locality) meaning some users are having much worse experiences than others.
Does this seem like a valid concern?
I haven’t done any actual experiments to validate the above - I’m hoping this is something the people here have thought about and you already have good answers.
kotlinx.coroutines have implemented their own thread pool a while ago (available under the Dispatchers.Default dispatcher) and they are doing a semi-FIFO scheduling. This is not the full FIFO you are looking for but it is a bit better, this pool also implement a work stealing mechanism.
If fairness is a great concern for your use case I would suggest creating an actor running in a newSingleThreadContext, as channels are fair you could send tasks and be sure they are executed fairly, but that would imply a quite high overhead.
In my experience their custom thread pool works quite well and the semi-FIFO scheduling never lead to a request standing by for a long time, I think there is plenty fields to improve in a codebase before worrying about the scheduling order.