Fast file read in Kotlin/Native

Hi!

Does anybody knows the faster way to read lines from huge file in Kotlin Native? I am using Posix fgets() and it works way slower than JVM methods or C++ streams or basically everything else.

I was considering to try to link Boost but it seems to be overkill.

Thanks!

Hello, @nikita.firsin! Maybe this library could help?

Artem, thank you!

I will try and respond here.

The suggested library didn’t help. It works a bit slower than pure fgets()-based approach and actually uses POSIX calls under the cover.

I believe that fast file reading could be implemented via memory-mapping or something like that.

Have you tried to set the stream handle to fully buffered (_IOFBF) and assigned a big buffer (e.g. 64K) with setvbuf() before doing any IO on it. See https://pubs.opengroup.org/onlinepubs/9699919799/functions/setvbuf.html for a full description.
If you read from STDIN for example the stream will most likely be read line buffered (_IOLBF) by default.

@juerei Thanks for pointers, using setvbuf() I have received 15% speedup, from 11 seconds to 9.

These numbers above are stable, but I observe that sometimes JVM reads this file in 4 seconds, I believe this is somehow related to OS caching.
Anyway, I think that 9 seconds is ok to read lines from 1.2 Gb file though, so I won’t dig further.

Thanks for help!