Using Kotlin in Apache Spark


#1

Hi

I am evaluating Kotin as my next language :slight_smile:

I want to know, any tried of using Kotin in Apache Spark project? Any experience you can share? Will it be very difficult to setup?

Thanks.


#2

Personally no, but it seems a lot of others have given it a spin: https://github.com/search?utf8=✓&q=kotlin+spark&type=Repositories&ref=searchresults


#3

I wrote a blog post on this recently. It works out-of-the-box thankfully. I think a lot more folks would use it with Kotlin if there was documentation on it.


#4

There’s a nice followup in https://thegmariottiblog.blogspot.de/2017/05/lets-make-apache-spark-more-kotlinesque.html

However, the github query from @mplatvoet suffers a lot from the fact that there’s a web-dsl project called https://github.com/perwendel/spark-kotlin . So it is a pain to search for kotlin + apache spark solutions. Not sure what motivated them to name their project after one of the most popular java-frameworks…

I was also wondering if there’s a apache-spark-shell available for kotlin?


#5

I use kt in spark SQL, it is a example : https://github.com/dushibaiyu/kotlin-spark-sql-es-read-example/