URL.readText, what do they mean by HUGE

Language Design
1

I am tempted to use the URL.readText command, but they say that it is nor recommended for “Huge Files”.

What do they mean by “Huge”? My URL could return JSON with a size of around 1500 KB. Is that considered Huge?

https://kotlinlang.org/api/latest/jvm/stdlib/kotlin.io/java.net.-u-r-l/read-text.html

2

It means that the entire response will be read into string, potentially consuming megabytes of memory. Whether it’s huge or tiny, depends on your use-case, if you’re running this code once on server with 128GB memory, it’s not an issue; if you’re running this code on tiny smartphone with 20 MB free RAM, it might be an issue.

If you are going to parse this JSON, you should use streams, there’s no need to keep the entire response in memory.

2 Likes
3

Moreover -on JVM- String is stored in memory using UTF-16 encoding, so 1.5M characters occupy 3M bytes (plus allocation of temporary buffers).

As rule of thumb avoid to use readText when you can use a stream.

1 Like