Can I use an InputStream as a body in Ktor HTTP client?

Libraries
1

Hello folks! I am stumped by something I have not been able to figure out myself despite considerable experimentation and googling.
In a nutshell, what I want to do is feed an InputStream as the body of a PUT request. I thought I could maybe use a ChannelWriterContent and provide a block that writes to the ByteWriteChannel but that was a bust. I can obviously readBytes() from the InputStream, but the stream can be arbitrarily large and I do not want to shove everything in memory.

Does anyone know of a way to achieve this?

Thanks!

1 Like
2

By the way I have managed to get this to work, but it feels very wrong

private suspend fun sendFile(data: InputStream, url: String) {
            client.use {
                it.put<String>(url) {
                    body = object : OutgoingContent.ReadChannelContent() {
                        override val contentType = OctetStream
                        override val contentLength = data.available().toLong()
                        override fun readFrom(): ByteReadChannel {
                            return data.toByteReadChannel()
                        }
                    }
                }
            }
    }

is there a better way?

3

I don’t think this is a bad solution. It does exactly what it should - it reads body from your input stream. I would probably verify that InputStream is automatically closed, but I guess it is.

Anyway, if for some reason you prefer to use ChannelWriterContent then you just need to write while reading - instead of reading the whole stream with readBytes(). You can do it manually, by creating a buffer, then reading and writing. You can also use existing copyTo() extension:

override suspend fun writeTo(channel: ByteWriteChannel) {
    inputStream.copyTo(channel)
}
1 Like
4

Thank you very much for your response! I tried your suggestion but it didn’t work, it looks like nothing is being received at the other end

it.put<String>(url) {
           body = object : OutgoingContent.WriteChannelContent() {
               override val contentType = OctetStream
               override suspend fun writeTo(channel: ByteWriteChannel) {
                   data.copyTo(channel)
               }
          }
}

Is this what you meant? I would have expected Ktor to do chunked transfer in the absence of a defined length, but it doesn’t look like it is happening :frowning:

5

Yes, this is what I meant.

I’m not any expert here, but I think it should work and actually it worked when I tested it. And yes, it uses chunked transfer:

User-Agent: Ktor client
Content-Type: application/octet-stream
Transfer-Encoding: chunked

d
Hello World!

0
6

Huh how bizarre, S3 tells me to get lost

2022-01-06T19:11:44.330142+00:00 [25644]: [DefaultDispatcher-worker-1] [INFO] io.ktor.client.HttpClient: METHOD: HttpMethod(value=PUT)
2022-01-06T19:11:44.331716+00:00 [25644]: [DefaultDispatcher-worker-1] [INFO] io.ktor.client.HttpClient: COMMON HEADERS
2022-01-06T19:11:44.333060+00:00 [25644]: [DefaultDispatcher-worker-1] [INFO] io.ktor.client.HttpClient: -> Accept: */*
2022-01-06T19:11:44.334316+00:00 [25644]: [DefaultDispatcher-worker-1] [INFO] io.ktor.client.HttpClient: -> Accept-Charset: UTF-8
2022-01-06T19:11:44.335466+00:00 [25644]: [DefaultDispatcher-worker-1] [INFO] io.ktor.client.HttpClient: CONTENT HEADERS
2022-01-06T19:11:44.336674+00:00 [25644]: [DefaultDispatcher-worker-1] [INFO] io.ktor.client.HttpClient: -> Content-Type: application/octet-stream
2022-01-06T19:11:44.960727+00:00 [25644]: [DefaultDispatcher-worker-1] [INFO] io.ktor.client.HttpClient: RESPONSE: 501 Not Implemented

S3 verbose error

 <Error><Code>NotImplemented</Code><Message>A header you provided implies functionality that is not implemented</Message><Header>Transfer-Encoding</Header><RequestId>xxxxx</RequestId><HostId>xxxxx</HostId></Error>"