I’ve just switched to Kotlin 1.3. Dropped GlobalScope here and there (yeah, yeah, I’ll refactor it later). Unfortunately, Default Dispatcher now creates tens of threads instead of 4 (# of CPU cores). CPU load has increased by about 200%, the extra time is spent in Thread.yield() (what?).
Could I have done something wrong to exhibit this non-obvious behavior? I’ll probably have to switch to newFixedThreadContext dispatchers, although I’m actually pretty happy with the idea of using the default one for most tasks.
Edit: Might be caused by recent changes in Ktor.io.
Edit 2: Might be caused by the loss of control over HttpClient’s dispatcher (it uses DefaultDispatcher now).
Edit 3: I’ve replaced GlobalScope with MyGlobalScope in my code and replaced CIO-based HttpClient with Apache-based. Some DefaultDispatcher threads are gone. Performance has improved a lot, but those DefaultDispatcher threads, which seem to do nothing but park(), are still giving me a headache.
Could I have done something wrong to exhibit this non-obvious behavior?
No, this is rather a framework issue. What I can recommend is to avoid CIO engine in Ktor for a while, it is in experimental status and its performance in an idle state is yet to be tuned. And as you can observe, Ktor CIO uses two instances of such dispatcher ad that does not help as well.
About why DefaultDispatcher consumes a lot of CPU: it was optimized for either heavy-load (when CPU is utilized in 80-100%) or very low load (close to zero with occasional IO): it adaptively spins when it is out of work and park only after some delay (parking and unparking is expensive), thus burning CPU for a while. It is much performant in a high load cases, but pays for it with increased CPU consumption in idle state.
There definitely could be some improvements and knobs in the dispatcher. Could you please report some sample project where you experience an unexpected CPU consumption due to default dispatcher to Issues · Kotlin/kotlinx.coroutines · GitHub?
Hi, I’m afraid I’d have to give out the whole project — not too enthusiastic about that. But you are right, my use case probably isn’t supported well currently. I have small bursts of work, where the server needs to do a lot of IO and CPU work, and then there’s a larger (but still less than 1s) period of silence, where everything needs to instantly settle down. Is there any switch I can use to tell DefaultDispatcher not to spin the threads as long?
Hi, yes, you can use system property kotlinx.coroutines.scheduler=off in order to switch default dispatcher to ForkJoinPool. But in Ktor that dispatcher is used unconditionally, so the only option is to switch CIO engine to Netty or Jetty.
