Memory Leak (and Crash)

We’re in the process of rewriting a large chunk of common business logic on both Android and iOS. In order to reduce development time we’re exploring the option of using Kotlin and Kotlin Native for the common logic. At this point, my biggest concern is a memory leak (and eventual crash) with a very simple example program.

Our “expect” class defines NetworkInterface as:

    expect class NetworkInterface() {
        fun execute(request: NetworkRequest, onCompletion:(NetworkResponse<ByteArray>) -> Unit)
    
        companion object {
            val instance: NetworkInterface
        }
    }
    
    data class NetworkResponse<DataType> ( val httpCode:Int, val data:DataType )
    
    data class NetworkRequest(val method: Method = Method.GET, val url: String) {
        enum class Method { GET, PUT, POST }
    
        fun execute(onCompletion:(NetworkResponse<ByteArray>) -> Unit) =
            NetworkInterface.instance.execute(this, onCompletion)
    }

While our actual implementation is:

    package com.greenwing.sample.core
    
    import platform.Foundation.*
    import konan.worker.*
    import platform.darwin.*
    import kotlinx.cinterop.*
    
    actual class NetworkInterface actual constructor() {
        val session = NSURLSession.sessionWithConfiguration(
                NSURLSessionConfiguration.defaultSessionConfiguration(),
                null,
                NSOperationQueue().apply {
                    name = "Parallel Queue"
                }
        )
    
        actual fun execute(request: NetworkRequest, onCompletion:(NetworkResponse<ByteArray>) -> Unit) {
            val url = NSURL.URLWithString(request.url)
            val urlRequest = NSMutableURLRequest.requestWithURL(url!!)
            urlRequest.HTTPMethod = request.method.name
    
            println("Request:$urlRequest")
    
            // This code runs on calling thread, happens to be the UI thread
            session.dataTaskWithRequest(urlRequest) { data, response, error ->
                // This code runs on background thread, per the session
                konan.initRuntimeIfNeeded()
    
                val httpResponse = (response as? NSHTTPURLResponse)
                val bytes = data?.bytes?.let { it.readBytes(data.length.toInt()) } ?: ByteArray(0)
    
                val networkResponse = when (error) {
                    null -> when (httpResponse) {
                        null -> NetworkResponse<ByteArray>(-1, bytes)
                        else -> NetworkResponse<ByteArray>(httpResponse.statusCode.toInt(), bytes)
                    }
                    else -> NetworkResponse<ByteArray>(-1, bytes)
                }
    
                networkResponse.freeze()
    
                dispatch_async(dispatch_get_main_queue()) {
                    konan.initRuntimeIfNeeded()
                    onCompletion(networkResponse)
                }
            }.resume()
        }
    
        actual companion object {
            actual val instance:NetworkInterface by lazy { NetworkInterface() }
        }
    }

Which is then called from the Swift UI thread using:

        GSCNetworkRequest(method: .get, url: "http://samples.openweathermap.org/data/2.5/weather?zip=94040,us&appid=b6907d289e10d714a6e88b30761fae22")
            .execute { response in
                print("response: \(String(describing: response))")
                
                switch(response.httpCode) {
                case 200..<299: self.available?.text = "Success"
                default:        self.available?.text = "Failure: \(response.httpCode)"
                }
                return GSCStdlibUnit()
            }

This has a fairly steady leak through initRuntimeIfNeeded as NSOperationQueue (particularly GCD queues) will randomly discard and create threads based on usage. Eventually (randomly varying from minutes to an hour or more) the process will actually crash, seemingly with various bugs indicative of memory corruption.

Ultimately we’re planning on using some kind of a Worker/Task model to wrap the underlying task mechanisms available on iOS and Android with the following requirements:

  • Some tasks should execute in the background and we may have multiple tasks simultaneously executing (a parallel dispatch queue)
  • Some tasks should execute serially on a specific serial dispatch queue to insure that only one is ever being processed at any given time
  • Obviously, some tasks should execute on the main thread, specifically for UI updates

On to the questions:

  1. Is there a better way to handle this multitasking issue, particularly given some required level of cooperation with NSOperationQueue/dispatch_queue to support NSURLSessionDataTask usage?
  2. Are there plans to fix the memory leaks with initRunTimeIfNeeded (presumably by tearing down memory when the associated thread exits, or by using some other mechanism)
  3. If 2 is out of the question, is there a way around needing to use initRunTimeIfNeeded?
  4. Can we pretty please get a void return type, so we don’t have to return StdlibUnit(), looks ugly as hell.

On the memory leak. You create on a class level (through the companion object, and lazily initialised) a global network interface. This interface is shared across all threads. Threads work with a shared memory model so when the thread disappears this interface doesn’t. You may instead want to use a thread local variable. It appears that the retention of this network interface is mainly some form of caching. Do you need to cache it? And if you do, is this the right place? (and can the interface handle being reused this way). You probably want to have some more generic cache implementation that can actually forget and provide different objects for different uses.

Ultimately there will be a lot of configuration information in the interface that will be moderately expensive to recreate, so whether it follows a “singleton model” or is just a single instance passed around is definitely the most appropriate model. In any case, at this point, it’s currently only accessed from the main thread, so there shouldn’t ever be more than one of them anyway.

Perhaps the NSURLSession could be thread local instead of owned by the NetworkInterface but again, there’s only one interface, so only one session, so it really doesn’t explain the memory leak I’m seeing.