Hi!
Does anybody knows the faster way to read lines from huge file in Kotlin Native? I am using Posix fgets() and it works way slower than JVM methods or C++ streams or basically everything else.
I was considering to try to link Boost but it seems to be overkill.
Thanks!
Hello, @nikita.firsin! Maybe this library could help?
Artem, thank you!
I will try and respond here.
The suggested library didn’t help. It works a bit slower than pure fgets()-based approach and actually uses POSIX calls under the cover.
I believe that fast file reading could be implemented via memory-mapping or something like that.
Have you tried to set the stream handle to fully buffered (_IOFBF) and assigned a big buffer (e.g. 64K) with setvbuf() before doing any IO on it. See setvbuf for a full description.
If you read from STDIN for example the stream will most likely be read line buffered (_IOLBF) by default.
@juerei Thanks for pointers, using setvbuf() I have received 15% speedup, from 11 seconds to 9.
These numbers above are stable, but I observe that sometimes JVM reads this file in 4 seconds, I believe this is somehow related to OS caching.
Anyway, I think that 9 seconds is ok to read lines from 1.2 Gb file though, so I won’t dig further.
Thanks for help!