How to avoid callback hell or "pyramid of doom"?


#1

When you start to create a real Javascript application (which gets data from the server using asynchronous XMLHttpRequest) you will easily end up with callbacks within callbacks and duplicated code.

http://callbackhell.com/

Does Kotlin by nature offer any remedy for this? Is the Yield feature coming soon?

Or can I only overcome this by finding a Promise/Future library?

Any recommendations?

Thanks

Jørund


#2

Hi Jørund!

> Does Kotlin by nature offer any remedy for this? Is the Yield feature coming soon?
Not soon and not before 1.0, but I think someday we’ll implement yield and/or async

Feel free to vote or star to get updates.

> Or can I only overcome this by finding a Promise/Future library?
> Any recommendations?

I believe that common logic should be extracted to common place :slight_smile:
Note that You can extract it to local functions instead of top level or class level. Additionally You can simplify your code with callable references.


#3

Thanks for the update!

When is the 1.0 coming out?


#4

When it's ready. Giving ETAs for programming languages does not quite work, unfortunately.


#5

:) No problem.


#6

TL;DR: I'd definitely recommend using RxJava & RxKotlin.

To me, the two main problems of the “callback hell” in NodeJS is the verbosity of the function syntax and the abscence of proper error handling.
ES6 will correct both problems with the lambda syntax and the standardized Promise library.

As fo Kotlin, the first problem is corrected by the language itself.

// Kotlin
downloadFile(url) { it.length }

// JS (ES 5)
downloadFile(url, function(file) { return file.length; });


<Digression>
Of course, the integration of a feature like async / yield would be tremendous but it needs a lot of work before it can be done. Having a look at how Dart integrated these in its latest version is very instructive. For Kotlin to have such a feature, it would need to:

  • Support for async functions.
  • Rely on a Future like object for such functions.
  • Integrate nicely with Java, as it is one of Kotlin’s primary goal.
  • Integrate nicely with usual thread based asynchronicity, we’re not going to rewrite all I/O libraries.
  • implement a “coroutine” runtime inside the JVM. Easier said than done since the JVM is not asynchronous like NodeJS or Dart. That’s the hardest part, the JVM is just not thought for that.

I would be immensly greatful to the Kotlin team should they do it, but I don’t think it will be done tomorow.
For starter, if we could have a C# like yield (wich is an “intermediate return”, nothing to do with async), that would be great (and not that complicated, I think).

</Digression>

For error handling, I recommend using RxJava & RxKotlin. The problem with NodeJS like callback is that you have to check for error handling in each callback. Using RxJava & RxKotlin allows you to subscribe / chain / combine Observables (Future / Promise like superset) and handle any error only once.
RxJava is a real pain to learn and properly understand, but once you’re past learning it, it is amazing how asynchronicity becomes easy. Especially in Kotlin.

Furthermore, the Observable API provides a feature that is comparable to yield:
 

val obs = observable<Int> {
  for (i in 15)
  it.onNext(i)
  it.onCompleted()
}

This will create an observable that will return all integers from 1 to 5 before completing. The execution will not hang at each call to onNext and all results will be stored waiting to be consumed, so it's not yield per se, but it's the closest we have now.

Hope it helps :wink:


#7

Thanks a lot for the input. I wills study your suggestions carefully as soon as I get the time.


#8

I guess you're talking about the case when compiling Kotlin to Javascript itself, right? Because on the JVM you'd just use threads.


#9

Hi Salamon!

I’m trying to use RxKotlin in my Kotlin (Javascript) project by adding the dependency:

 
<dependency>
  <groupId>io.reactivex</groupId>
  <artifactId>rxkotlin</artifactId>
  <version>0.21.0</version>
</dependency>
 

However I’m struggling to use anything from rx package in my project. Is this enough to start using it?


Screen Shot 2015-05-18 at 15.58.48.png (3.23 KB)

#10

I've never used kotlin to compile to JS but my understanding is that the kotlin compiler compiles Kotlin code to Javascript, and not JVM Bytecode. So, as I understand it, you can't use RxKotlin in JS, nor any Jar compiled library for that matter. Besides, RxKotlin & RxJava are using threads, which is not supported by JS.


#11

You can't use rxkotlin with JavaScript. The main reason is that RxKotlin is targeted to RxJava implementation, not to RxJS. So RxKotlin is JVM library compiled to bytecode. To use RxJS with Kotlin you need corresponding Kotlin bindings. We don't have it yet and it is not planned for now. You always can make these bindings.To get started you can look how JavaScript native declarations made in kotlin-js-library.

RxJS bindings may look like:

native trait Disposable {
    fun dispose()

  companion object Instance {
  fun create(onDispose: () -> Unit): Disposable
  fun isDisposable(d: Any): Boolean
  val empty: Disposable
  }
}

native trait Observable<T> {
  fun subscribe(onNext: (T) -> Unit, onError: (Throwable) -> Unit, onComplete: () -> Unit) : Disposable

  companion object Instance {
  fun <T> asObservable(vararg args : T) : Observable<T>
  }
}

native object Rx {
  val Observable: Observable.Instance
  val Disposable: Disposable.Instance
}

and so on

so you will be able (not tested) to do:

fun main(args: Array<String>) {
    val myObservable = Rx.Observable.asObservable(1, 2, 3)

  myObservable.subscribe(
           onNext = {
           console.log(“Got next”, it)
           },
           onError = {
           console.log(“Failed”, it)
           },
           onComplete = {
           console.log(“Completed”)
           }
  )
}


#12

Not really an answer, but I'm working on Kovenant which is a promises library for Kotlin.

At the time of writing there is no JS implementation (only a JVM) but I’m keeping the API runtime agnostic so other runtimes can be supported in the future.
I haven’t scheduled to do so yet though.


#13

That looks really nice Mark! I've been hoping someone would do a really great futures/promises library for Kotlin. I am slowly converting an existing codebases that uses a mix of Guava ListenableFuture and Java 8 CompletableFuture. The ListenableFuture's have to stay because they come from a Java library, but I hope that I can find a way to glue them to Kovenant and then scrap the horrible Java 8 API completely.


#14

Glueing them to Kovenant is quite straight forward, see this example with CompletableFutures. Code for Guave should be similar.


#15

Hi Mark,

It seems to me this can be trivially achieved with Rx (Java or Kotlin) with the added benefit of composition and a lot operators being already implemented for you.

What’s the added value of Komponents over Rx?


#16

It's hard not to turn the answer into some "x versus y" post so I try to keep it to a level where I think goals simply differ.

Rx targets anything that runs on the JVM where Kovenant targets Kotlin.
This means that Kovenant is designed in a way that it really levarages Kotlin’s language constructs. E.g. in Kotlin you ideally have one closure per function as the last parameter as opposed to multiple closures per function.
Furthermore the API of Kovenant is free from JVM specific classes and is open for implementations on other runtimes like JavaScript.

Garbage matters
Although we have the luxury of our object instances being managed, garbage can seriously degrade overal performance on certain platforms, most notably Android. So Kovenant tries to limit the overhead, and thus the garbage, as much as possible.

Classcount matters
There’s lightweight and there’s lightweight. The benefit you mention “of composition and a lot operators being already implemented for you” is a con on a platform like Android where class and method count really matters (since there is a hard limit).

Non blocking
Kovenant is written entirely with non-blocking algorithms which can matter if you expect a lot of thread collisions.

It’s of course up to you whether these things matter (enough). At the and it’s all just flavor.


#17

One thing I just noticed is there's no way to do a blocking get of a promise, in Kovenant. I see there's a bug filed for it in YouTrack (for some reason I can't use my jetbrains account to log in, though it says I can).

A blocking get() is quite useful for the case where you have a sequence of operations and some can be parallelised and others cannot, but the overall operation must be blocking. For example, a parallel foreach. It’s also highly useful when an API wishes to provide both an async and blocking API: your method can just return a future/promise, and if the user wants a blocking version of the API they just stick .get() afterwards and they have what they want.


#18

I'm going to support it. It's plannend to be part of the next minor release (2.2.0) (will do a patch version (2.1.1) first to fix two bugs) It's actually quite trivial to implement but at the cost of an extra object instance per promise instance. The only way to avoid that is using plain Java and use `sun.misc.Unsafe` which I wan't to avoid for runtime compatibility reasons.

That being said I don’t think, if I understand you correctly, that get() is necessary for the use cases you’re adressing. Just to say it shouldn’t stop you :slight_smile:
I’m experimenting with an asynchronous foreach.
A blocking get can also be implemented as described in the docs which is dedicated to blocking functionality.

But like I said it’s going to be in the next minor version, simply because developers expect it.


#19

Right, it can of course be done with other objects like a latch (or indeed, a CompletableFuture). But it's indeed the purpose of a promises API to avoid having to create extra objects like that :-)

I think the async-or-sync API use case is reasonably compelling. I often write code that I wish to use async in a case where, e.g. there is a GUI that must be responsive, but also in a command line tool where blocking the main thread isn’t a problem so it’s easier to write straight line synchronous code. Being able to easily do a blocking get on a promise makes both scenarios easy.

So, good to hear it’s coming! Looking forward to it.


#20

Perhaps there's a better place to discuss Kovenant than here, but I have another question. You gave promises a type parameter for the failure object case, which makes specifying a promise more verbose. I haven't seen this before. Is this to do with trying to be JVM neutral? Are there so many use cases for non-Throwable-descended failure objects? I don't believe I've ever seen a promise that requires this to be specified before.

One more thing: It’d be neat if deferred().resolve()/reject() returned the underlying promise instead of Unit. In my app I have the following use case:   an operation that loads a file from disk, and then does some long running operationg (network lookups) based on the content. I have a method that does all this that returns a promise. The obvious solution is wrap everything in async{}, however, I would like to do the disk IO in a blocking manner (this is not an Android app) because that way issues like the file being unreadable for some reason result in an immediate error message rather than a spinner flashing on screen as the async operation completes almost but not entirely immediately.

I know normally blocking the GUI thread to do disk IO is an issue, especially on Android, but for my desktop app it keeps things simpler in some cases. So right now I have code like:


fun foo(path: Path): Promise<Thing, Exception> =
  try {
  val thing = loadThing(path);
  async {
  checkThing(thing)
  }
  } catch (e: Exception) {
  val d = deferred<Thing, Exception>()
  d.reject(e)
  d.promise
  }

It'd be more convenient if the last bit could just be

  deferred<Thing, Exception>().reject(e)