Using Kotlin in Apache Spark

1

Hi

I am evaluating Kotin as my next language :slight_smile:

I want to know, any tried of using Kotin in Apache Spark project? Any experience you can share? Will it be very difficult to setup?

Thanks.

2

Personally no, but it seems a lot of others have given it a spin: Search · kotlin spark · GitHub

3

I wrote a blog post on this recently. It works out-of-the-box thankfully. I think a lot more folks would use it with Kotlin if there was documentation on it.

1 Like
4

There’s a nice followup in Guido Pio Mariotti: Let's make Apache Spark more Kotlinesque

However, the github query from @mplatvoet suffers a lot from the fact that there’s a web-dsl project called GitHub - perwendel/spark-kotlin: A Spark DSL in idiomatic kotlin // dependency: com.sparkjava:spark-kotlin:1.0.0-alpha . So it is a pain to search for kotlin + apache spark solutions. Not sure what motivated them to name their project after one of the most popular java-frameworks…

I was also wondering if there’s a apache-spark-shell available for kotlin?

5

I use kt in spark SQL， it is a example : GitHub - dushibaiyu/kotlin-spark-sql-es-read-example: use spark sql read data from Elasticsearch use kotlin

1 Like
7

hello
are you able to provide working spark-submit command line to run your git solution please ?

thanks

8

Just in case anyone is still listening to the thread: GitHub - Kotlin/kotlin-spark-api: This projects gives Kotlin bindings and several extensions for Apache Spark. We are looking to have this as a part of Apache Spark 3.x