Promise coroutines: await doesn't actualy wait

HughG · May 13, 2017, 5:28pm

Hi all,

I’ve just applied the async/await implementation from Using coroutines to avoid callback hell when using XMLHttpRequest to my toy project, in #12 Refactored to use continuations, but ... · HughG/partial-order@2e4e2d4 · GitHub.

It looks much nicer than before, but the calls to Promise<T>.await don’t seem to actually wait; that is, they don’t prevent the rest of an async block from executing immediately.

This is all using

IntelliJ 2017.1.2
Kotlin 1.1.2_2 (compiler and plugin)
pouchdb 5.3.1
CouchDB 1.6.1
jquery 3.2.1 with Kotlin API from ts2tk
Firefox Developer Edition 54.0a2 64-bit
Windows 10

With the refactoring in that commit, the (browser console) output of my example is like this:

Object {  }  kotlin-test-app.js:129:13
Loading graph ...  kotlin-test-app.js:311:13
Nodes by rank ...  kotlin-test-app.js:222:5
TypeError: graph.ranks is undefined: listByRank@file:///C:/Users/hughg/Documents/dev/partial-order/web/js/app/kotlin-test-app.js:223:9
this["kotlin-test-app"]</Coroutine$main$lambda.prototype.doResume@file:///C:/Users/hughg/Documents/dev/partial-order/web/js/app/kotlin-test-app.js:189:13
CoroutineImpl.prototype.doResumeWrapper_0@file:///C:/Users/hughg/Documents/dev/partial-order/web/js/app/lib/kotlin.js:3805:20
CoroutineImpl.prototype.resume_11rb$@file:///C:/Users/hughg/Documents/dev/partial-order/web/js/app/lib/kotlin.js:3794:5
startCoroutine_0@file:///C:/Users/hughg/Documents/dev/partial-order/web/js/app/lib/kotlin.js:28974:5
async$lambda/<@file:///C:/Users/hughg/Documents/dev/partial-order/web/js/app/kotlin-test-app.js:1528:7
async@file:///C:/Users/hughg/Documents/dev/partial-order/web/js/app/kotlin-test-app.js:1532:12
main@file:///C:/Users/hughg/Documents/dev/partial-order/web/js/app/kotlin-test-app.js:218:17
this["kotlin-test-app"]<@file:///C:/Users/hughg/Documents/dev/partial-order/web/js/app/kotlin-test-app.js:1695:3
@file:///C:/Users/hughg/Documents/dev/partial-order/web/js/app/kotlin-test-app.js:4:27
  kotlin-test-app.js:1553:5
Using //@ to indicate sourceMappingURL pragmas is deprecated. Use //# instead[Learn More]  kotlin-test-app.js:1699
Object {  }  kotlin-test-app.js:129:13
GET 
XHR 
http://localhost:5984/ranking/ [HTTP/1.1 404 Object Not Found 0ms]
The above 404 is totally normal. PouchDB is just detecting if the remote exists.

However, at the previous commit, it looks more like this:

Using //@ to indicate sourceMappingURL pragmas is deprecated. Use //# instead[Learn More]  kotlin-test-app.js:1202
GET 
XHR 
http://localhost:5984/ranking/ [HTTP/1.1 404 Object Not Found 2ms]
The above 404 is totally normal. PouchDB is just detecting if the remote exists.  pouchdb-5.3.1.js:5800:5
Object { db_name: "ranking", doc_count: 0, doc_del_count: 0, update_seq: 0, purge_seq: 0, compact_running: false, disk_size: 79, data_size: 0, instance_start_time: "1494695881360340", disk_format_version: 6, 4 more… }  kotlin-test-app.js:43:7
Bulk store inputs:  kotlin-test-app.js:920:7
Object { _id: "N_github:partial-order/HughG/14", type: "N", description: "Gradle task for ts2kt" }  kotlin-test-app.js:924:9
Object { _id: "N_github:partial-order/HughG/13", type: "N", description: "Integrate NodeJS packages into Grad…" }  kotlin-test-app.js:924:9
...
Bulk store results:  kotlin-test-app.js:936:7
Object { ok: true, id: "N_github:partial-order/HughG/14", rev: "1-dcfc41c1ca0f9be813c95c3982cc27c8" }  kotlin-test-app.js:940:9
Object { ok: true, id: "N_github:partial-order/HughG/13", rev: "1-e85cb281e1f5cc08ee612099926c0a37" }  kotlin-test-app.js:940:9
...
Loading graph ...  kotlin-test-app.js:178:5
Nodes:  kotlin-test-app.js:123:7
Object { total_rows: 13, offset: 0, rows: Array[13] }  kotlin-test-app.js:124:7
Node N_github:partial-order/HughG/10: Investigate Yested for UI  kotlin-test-app.js:136:11
Node N_github:partial-order/HughG/11: Support pagination when fetching from GitHub  kotlin-test-app.js:136:11
...
Edges:  kotlin-test-app.js:155:7
Object { total_rows: 13, offset: 0, rows: Array[0] }  kotlin-test-app.js:156:7
Loading graph ... done.  kotlin-test-app.js:172:7
Nodes by rank ...  kotlin-test-app.js:80:5
Caching ranks ...  kotlin-test-app.js:525:7
0 0 'Investigate Yested for UI' <- 'null'  kotlin-test-app.js:515:7
1 0 'Support pagination when fetching from GitHub' <- 'null'  kotlin-test-app.js:515:7
...

In the Kotlin source, after the refactor, listByRank is failing because it’s trying to access graph.ranks when the graph hasn’t been initialised. In fact, the db hasn’t even been initialised: it outputs Object { } near the start where previously it output Object { db_name: "ranking", doc_count: 0, doc_del_count: 0, update_seq: 0, purge_seq: 0, compact_running: false, disk_size: 79, data_size: 0, instance_start_time: "1494695881360340", disk_format_version: 6, 4 more… } kotlin-test-app.js:43:7.

Any clues, from anyone willing to check out and dig through my source code? If there are no takers for that, I can try to boil it down to a small repeatable case. I did try to dig into the generated JavaScript but in the presence of coroutines it’s pretty impenetrable.

Alexey.Andreev · May 13, 2017, 7:25pm

Sorry, I think nobody can help you until you provide source code that reproduces the issue. The only thing I can tell is we fixed some issues in JS corotuines recently (see KT-17446, KT-17067, KT-17281, KT-16951 and KT-16658). Please, look at these issues and try to find out if you are using some of the provided code patterns that cause errors. Another option to grab latest build of Kotlin compiler from TeamCity and compile against it and see if this solves the issue (also, you can wait until we publish 1.1.3-EAP).

HughG · May 20, 2017, 11:26am

Hi @Alexey.Andreev,

that’s a totally reasonable response. I was info-dumping after a frustrating few hours last week but, if I got a bug report, I’d be asking for an isolated repeatable case, too

So, I upgraded to IJ2017.2 EAP and Kotlin plugin, as described here: Trouble with EAP - #2 by HughG 1.1.3-eap-34-IJ2017.2-1. That didn’t make my situation any better, so I cut it down to the following test case. I freely admit that I don’t really know what I’m doing in terms of using coroutines to implement async/await, I’m just copying others’ code, as follows.

package org.tameter.ksandbox

import kotlin.browser.window
import kotlin.coroutines.experimental.*
import kotlin.js.Promise

// From https://youtrack.jetbrains.com/issue/KT-17067
private object JavaScriptContext : AbstractCoroutineContextElement(ContinuationInterceptor.Key), ContinuationInterceptor {
    override fun <T> interceptContinuation(continuation: Continuation<T>) = object : Continuation<T> {
        override val context = continuation.context

        override fun resume(value: T) {
            window.setTimeout({ continuation.resume(value); }, 0)
        }

        override fun resumeWithException(exception: Throwable) {
            window.setTimeout({ continuation.resumeWithException(exception) }, 0)
        }
    }
}

// Also from https://youtrack.jetbrains.com/issue/KT-17067
fun <T> immediateAsync(c: suspend () -> T) {
    c.startCoroutine(object : Continuation<T> {
        override fun resume(value: T) { }

        override fun resumeWithException(exception: Throwable) { throw exception }

        override val context = JavaScriptContext
    })
}

// From https://discuss.kotlinlang.org/t/using-coroutines-to-avoid-callback-hell-when-using-xmlhttprequest/2450/3
fun <T> promiseAsync(c: suspend () -> T): Promise<T> {
    return Promise { resolve, reject ->
        c.startCoroutine(object : Continuation<T> {
            override fun resume(value: T) = resolve(value)

            override fun resumeWithException(exception: Throwable) = reject(exception)

            override val context = EmptyCoroutineContext
        })
    }
}

// Also from https://discuss.kotlinlang.org/t/using-coroutines-to-avoid-callback-hell-when-using-xmlhttprequest/2450/3
// Should work with promiseAsync; maybe doesn't make sense to use it with immediateAsync, I'm not sure.
inline suspend fun <T> Promise<T>.await() = suspendCoroutine<T> { c ->
    then({
        console.log("Resolving with $it")
        c.resume(it)
    }, {
        console.log("Rejecting with $it")
        c.resumeWithException(it)
    })
}

suspend fun <T> promise(value: T): Promise<T> {
    return Promise.resolve(value)
}

fun main(args: Array<String>) {
    promiseAsync {
        console.log("1")
        val a = promise("aardwolf").await()
        console.log(a)
        console.log("2")
    }
}

So, if I run the above code as it stands (in Firefox) I get the following console output.

1  sandbox.js:160:13
Object {  }  sandbox.js:173:13
2  sandbox.js:174:20
Resolving with aardwolf  sandbox.js:113:7

but I was hoping for

1
aardwolf
2

If I have main call immediateAsync I get this:

1  sandbox.js:160:13
Object {  }  sandbox.js:173:13
2  sandbox.js:174:20
Resolving with aardwolf  sandbox.js:113:7
TypeError: this$await.then is not a function[Learn More]  sandbox.js:125:7
	await$lambda/< file:///C:/Users/hughg/Documents/dev/partial-order/web/js/sandbox/sandbox.js:125:7
	suspendCoroutine$lambda/< file:///C:/Users/hughg/Documents/dev/partial-order/web/js/sandbox/lib/kotlin.js:29004:7
	sandbox</Coroutine$main$lambda.prototype.doResume file:///C:/Users/hughg/Documents/dev/partial-order/web/js/sandbox/sandbox.js:171:29
	CoroutineImpl.prototype.doResumeWrapper_0 file:///C:/Users/hughg/Documents/dev/partial-order/web/js/sandbox/lib/kotlin.js:3164:20
	CoroutineImpl.prototype.resume_11rb$ file:///C:/Users/hughg/Documents/dev/partial-order/web/js/sandbox/lib/kotlin.js:3153:5
	JavaScriptContext$interceptContinuation$ObjectLiteral$resume$lambda/< file:///C:/Users/hughg/Documents/dev/partial-order/web/js/sandbox/sandbox.js:30:7

Am I just doing something dumb here, or missing some simple step?

Alexey.Andreev · May 22, 2017, 10:36am

Thanks for your example. It needs more deep investigation. In case of a bug in the compiler, I’ll post an issue on YT with detailed explanation and workaround and publish link here.

Alexey.Andreev · May 24, 2017, 1:44pm

Looks like compiler generates wrong code for inline functions that only call suspendCoroutine. I’ll post issue after some investigation. As a workaround you can remove inline modifier from the following line:

inline suspend fun <T> Promise<T>.await() =

Also, I recommend you to remove `suspend modifier from the following line:

suspend fun <T> promise(value: T): Promise<T>

Alexey.Andreev · May 24, 2017, 2:32pm

https://youtrack.jetbrains.com/issue/KT-18063

HughG · May 26, 2017, 6:01pm

@Alexey.Andreev, glad I could help you find a bug, but removing “inline” doesn’t solve the problem in my real code. So, I suppose I need to create a more realistic example. My real code is calling into PouchDB and I don’t know what it does internally, but I guess I’ll have to dig into it.

Topic		Replies	Views
Using coroutines to avoid callback hell when using XMLHttpRequest JavaScript	6	5427	May 13, 2017
Async await on the client (JavaScript)	4	9333	October 12, 2017
Kotlin Coroutines are super confusing and hard to use Language Design	6	9769	December 6, 2018
Server-side async/await with coroutines	12	5356	January 24, 2018
Recommendations for using coroutines in Kotlin JS JavaScript	1	6484	March 22, 2018

Promise coroutines: await doesn't actualy wait

Related topics