I posted this on StackOverflow (https://stackoverflow.com/q/55131122/1431244) and it was suggested I might have more luck posting here:
As I understand it, by default, if you start a Kotlin Coroutine via
async it’ll launch in
CommonPool (or if you use
GlobalScope ). And
CommonPool is a
ForkJoinPool and that, by default, is in non-async mode so it executes tasks in LIFO order. That seems like a poor choice for something like asynchronous web server applications where we’d want fair scheduling: we don’t want the poor sucker who hit our web server first to wait for all calls that came later. I did run some stuff in a debugger and it seems that the co-routine scheduling is done by calling
execute on the
ExecutorService so I think there isn’t any Kotline stdlib code there to help out.
I know that if there are free threads in a
ForkJoinPool they’ll work steal and that can help, but I’d imagine that in a web server experiencing a burst of traffic that’s keeping all threads busy the LIFO nature of CommonPool would mean that metrics like 95th percentile latency would be worse (average latency might actually be better due to locality) meaning some users are having much worse experiences than others.
Does this seem like a valid concern?
I haven’t done any actual experiments to validate the above - I’m hoping this is something the people here have thought about and you already have good answers.