Immutable?

1

Hi,

just had a bit a look at Fantom and I like the idea to have immutable classes. This is very helpful when doing concurrent programming to be on the safe side that concurrent threads don’t change your data. I once spent about 2 years only reproducing race conditions and fixing them. I think I know how shared state can be cumbersome … There is no easy way to solve this easily, but immutable classes would help a great deal. Making instance variables of a class immutable does not help a great deal, though, IMHO. Exchange the whole chunk you want to send to the other thread in a read-only object and you are fine. Also forces you to think exactly what you want to exchange (design!) rather than just setting some inst var immutable. Just pondering some ideas :wink:

/Oliver

2

The question was of course whether it would make sense for immutable to be added to Kotlin ...

3

kotlin has that today using the 'val' keyword for values... 

// person is immutable
class Person(val name: String, val city: String) {
}

though we need the collection.toImmutable() or collection.toUnmodifiable() methods in the standard library (we don't have those yet). http://youtrack.jetbrains.com/issue/KT-2624

4

btw nothing on the JVM can ever be really immutable; as with reflection you can update private fields - even final ones - directly :)

5

Fantom's immutable (const) classes explicitly declare they are immutable and the compiler enforces it. The Fantom type system knows about immutability so the compiler can ensure you don't share mutable state between threads. It's a nice feature and I'd love to see it in Kotlin.

http://youtrack.jetbrains.com/issue/KT-1179

6

I guess one day we could add a const annotation that fails a build if a const-annotated class has a var inside it?

Not sure I grok the bit about the compiler ensuring you don’t share mutable state between threads though; how does it know when its OK to do so? (e.g. you’ve decided to use a thread safe object like a ConcurrentHashMap or something)? Immutable data structures are one way to solve concurrency; but there are many different approaches (e.g. STM, actors, java.util.concurrent stuff & compare & swap based algorithms to avoid synchronized locks etc. How would a compiler grok the transitive closure of objects & functions callable from different threads?

7

The compiler doesn't grok anything. You're right, that would be hard. But if you're writing a function that shares state between threads you can ensure that the parameters are immutable. That's what Fantom's actor framework does (it also allows you to send immutable objects by serializing them).

I was wrong about the Fantom compiler allowing you to specify immutability for function parameters, it’s a runtime error if you try to share a mutable object that’s not serializable. Although it would be great if it were possible to statically enforce immutability for parameters in Kotlin. Not sure how feasible that is though.

http://fantom.org/doc/docLang/Concurrency.html

8

kotlin has that today using the 'val' keyword for values...

Right, of course ...

Immutable data structures are one way to solve concurrency; but there are many different approaches (e.g. STM, actors, java.util.concurrent stuff & compare & swap based algorithms to avoid synchronized locks etc.

Immutable messages make message exchange between actors a lot more reliable. STM works well when there are more reads than writes. Compare & Swap is only for changing single values in one atomic operation. You can't do something like this:

synchronized {
  if(sharedVar == …) {
          otherSharedVar = …;
  }
}

But I agree that shared nothing as with actors is not always easy to achieve in practical. With actors you may not have threads that deadlock, but you may have a deadlock in the message flow. The actor hype guys don’t tell you that. I think the developer needs to have some tools to deal with concurrency at his disposal and is left on his own in deciding which ones are appropriate for a given situation.

9

You'd be amazed what you can do with CAS these days; you can build full lock free implementations of collections, maps and so forth. Hardcore CAS state machine stuff; the kind of thing you wanna reuse from a library :) http://code.google.com/p/jjoost/wiki/LockFreeHashStore

Immutabiliy and persistent collections (as used heavily in clojure et al) are certainly a nice & simple approach to concurrency & avoid lots of problems - they make things much easier to reason about for application programmers. Though for really high performance requirements; immutabiliy seems to add lots of cost onto the garbage collector (as every opteration requires allocating new objects), so CAS tends to be faster - especially for collections.

10

Yeah, CAS is an interesting idea and AFAIK hardware vendors like Intel have announced to add support for it with some new special instruction in their processors. This will probably make things much more efficient. Things like LockFreeHashStore are interesting as CAS is being made use of here for something more advanced than changing values atomically as in java.util.concurrent. As long as your synchronization problems can be confined to some abstract data type like LockFreeHashStore you are fine. However, sometimes you have objects running in their own threads (not necessarily actors, just think of different machines in a production plant that produce something in co-operation) doing some work together and need to get themselves co-ordinated that involves changing data in a shared store and the whole thing cannot be confined to an abstract data type like LockFreeHashStore any more (the synchronization issue is even application specific ...).

I like the idea of STM as well. It is not completely generic as it may lead to some kind of thrashing when the number of reads are not significantly higher than the number of writes. But STM may be an appropriate solution for many concurrency problems. Think Kotlin should provide some bridge f.ex. to DeuceSTM or Multiverse. If I had not started that many projects in the past that remained unfinished, because something more hip to work on came along, I would sit down and do it :-))).  Would also be nice if Kotlin had a little actor library. This would be a nice show case/“proof of concept” to show the power of Kotlin and a good litmus test that enables people to compare Kotlin quickly with what other languages can do. Something simple with an abstract actor class (or trait) and an actor thread pool as in Fantom would be a solid base to start with. I have something new to suggest, by the way … ;-). I think the idea of isolates as in Dart is a very clean and simple approach to make dealing with concurrency issues simpler.

Cheers, Oliver

11

We can also re-use akka & jactor (with maybe some nice extension methods to make then kooler for kotlin) for actors; am not sure if we must have actors in the standard library since there are many good actor libraries out there already.

I’d also like to see a kotlin version of gpars, particularly nice kool extension methods for the concurrent collection processing library using the JSR 166y fork/join framework (see the docs).

The nice thing about kotlin is we can reuse any JVM library; whether its for java/kotlin/scala and thanks to extension methods we can make it more kool (without having to wrap it).

12

We can also re-use akka & jactor

The question is whether you can write a wrapper for the akka Scala API with Kotlin. For example:

Equally, because of the magic of scala you could write this:

val orderHistory: Future[List[Order]] = for {
  id <- CustomerLookupActor.ask(customerName).mapTo[Int]
  details <- CustomerDetailsActor.ask(id).mapTo[CustomerDetails]
  history = details.orderHistory


} yield history


AFAIK, akka only really starts to fly with the Scala API. Being able to wrap the akka Scala API in Kotlin would be a good “proof of concept” for Kotlin … Eventually, the people that are really serious with actors would go with Scala, I fear, for the sake of akka.

13

That syntax is pretty cute. Scala's for statement allows for some powerful LINQ style compositions; plus allows 'push based' rather than "pull based" iteration too (e.g. calling collection.forEach(block) rather than pull based with collection.iterator().

It would be nice to see if we can come up with some kind of syntax in Kotlin for doing similar things; composing push-or-pull based iteration over Futures / collections / parallel collections in a simple way. Wonder if it could be combined with the try-with-resources kinda syntax (which in some ways is kinda similar, defining a number of variables with a function block composed into what looks like a single block)

One option is just to use map/flatMap in a chain though… (assuming we add a nice Future class like the one from guava/scala with nice collection like extension functions to the standard library so they are easy to compose). Something like this could be possible in Kotlin…

val orderHistory: Future[List[Order]] = CustomerLookupActor.ask(customerName).mapTo<Int>(javaClass<Int>()).   flatMap { id -> CustomerDetailsActor.ask(id).mapTo<CustomerDetails>(javaClass<CustomerDetails>()) }.   flatMap { details -> details.orderHistory }
The verbosity of the passing of the types into the mapTo() methods is ugly; hopefully we can fix that with the inline reified functions issue. Then it would be this...
val orderHistory: Future[List[Order]] = CustomerLookupActor.ask(customerName).mapTo<Int>().   flatMap { id -> CustomerDetailsActor.ask(id).mapTo<CustomerDetails>() }.   flatMap { details -> details.orderHistory }

Which isn’t so far from the Scala version; just a little different - in the same way LINQ tries to turn things around to be a bit more natural to programmers; more SQL-ish.

It might be worth experimenting with Akka / JActor and some standard kotlin extension functions on Future to see how clean we could get code like the above in kotlin. Composing collections, options & futures cleanly is a great use case for languages.

Coming back to the try-with-resources kinda syntax, if we had some function, say ‘forEach()’ for the push based iteration (which we have already for collections and could easily implement for closeable things or futures etc), I wonder if we could have a syntax statement something like this (I’m purposely avoiding ‘for’ here…)

// try with resources syntax
use ( i = FileInputStream(“foo.txt”)
  o = FileOutputStream(“bar.txt”) {
  doSomething(i, o)
}

which is just syntax sugar for
{   val i = FileInputStream("foo.txt")   i.forEach {   val o = FileOutputStream("bar.txt")   o.forEach {   doSomething(i, o)   }   } }
then we could use the same syntax sugar for composing collections & futures like in scala:
val orderHistory: Future[List[Order]] = use (   id = CustomerLookupActor.ask(customerName).mapTo<Int>()   details = CustomerDetailsActor.ask(id).mapTo<CustomerDetails>()   history = details..orderHistory ) yield history

I sneaked in a yield statement in there just to be able to do it :)  but you get the idea.

I do quite like the scala idea of having a single for statement that can represent multiple iterations in one statement and work in pull or push ways; but maybe in kotlin, being more explicit and less surprising, we’d need 2 different statements for .iterator() versus .forEach()? Afterall they can have performance impacts, so maybe a different syntax does make sense?

Thoughts?

14

Hi James,

interesting read. Reading your post I got a bit confused and therefore looked up some articles on the Internet to understand what yield in Scala really does. I must admit that I had no propper understanding of what “yield” is for in Scala. I thought it had something to do with the current thread voluntarily giving up the processor for some other thread or it means that some thread is spawned. I thought this had something to do with the Futures created (Future[List[Order]]). I was a bit too careless there, I guess, and too quick in making a post. Anyway, now I understand why collect has disappeared in stdlib and has been replaced with map ;-).

In that case this statement

val orderHistory: Future[List[Order]] = for {
  id <- CustomerLookupActor.ask(customerName).mapTo[Int]
  details <- CustomerDetailsActor.ask(id).mapTo[CustomerDetails]
  history = details.orderHistory

} yield history

is only about looping and has nothing to do with actors. Oops … The interesting part concerning actors is the Future[List[Order]] thing. But how to implement a Future is a library issue, not a language issue. The above example is probably not a good sample for the power of yield (see this article). A simple Smalltalk/Groovy-style collect iteration would do the same in that specific case.

Also, excessive looping is a hint for broken design (some classes or methods missing? lack of information hiding?). At least in OOP. Maybe not in functional programming  where this yield thing might have its place. To me, OOP, functional programming and Erlang all thrown together into one language is just three languages in one. But it does not create a meaningful fusion of those things to form a new coherent construct. To achieve this the Scala people would have had to think, f.ex., how Erlang can be combined with OOP and then bring the combination to Scala rather than just adding Erlang to it. Instead, to make it worse, they even added case classes to it, which you happily don’t need in OOP. That was when I lost my interest in Scala. Okay, I’m starting to make a digression …

Another issue is that exception handling in asynchronous programming is really hard, see “<a href=”http://www.lirmm.fr/~dony/postscript/exc-inbook06.pdf“>Exception Handling and Asynchronous Active Objects”  (see the section about Guards). IMHO Scala actors and akka don’t do much about it. Futures, supervision, fault-tolerance don’t solve this. It’s just something they’ve taken from Erlang. Those things are useful, but not sufficient. I was talking to them on the akka mailing list and one of the guys from Typesafe seemed to be interested. Let’s see. I have my own ideas from earlier when working on asynchronous control systems, but I’m a regular developer and not an exceptional one like you guys who have freedom to play with things at work they are interested in. I have to develop some software for some company and only can play with those things in my spare time and that is never enough for something really serious.

I think a better approach than using Erlang’s receive { … } clause as in Scala and other languages is to add closures to queues as with Grand Central Dispatch or HawtDispach. Erlang’s receive thing, IMHO, is some solution you have to stick to if you are forced to program with global functions and don’t have classes and instances. Adding closures to a queue that is served by its own thread gets around having to define case switches in the receive clause as in http://www.scala-lang.org/node/242 I think that would be something cool to work on :-).

Regards, Oliver

15

I found this good documentation page about Scala futures: http://docs.scala-lang.org/overviews/core/futures.html The relevant section starts with "Lets assume that we want to exchange US dollars for Swiss francs (CHF)." Think I now understand this piece of code:

val orderHistory: Future[List[Order]] = for {   id <- CustomerLookupActor.ask(customerName).mapTo[Int]   details <- CustomerDetailsActor.ask(id).mapTo[CustomerDetails]   history = details.orderHistory

} yield history


Yes, it’s need. But you can do the same thing with any language that has closures. No? I lately had a quick look at Erlang and realized that Scala/Akka is more or less an attempt to clone the ideas in Erlang. Problem is that Erlang is for OTP. So Akka is also geared towards OTP. But actors are not only for OTP, they are for concurrent programming in general. Actors geared towards Erlang-style OTP miss functionality required when developing asynchronous control systems for example (OTP = lots of data, not mich logic; control systems = little data, lots of logic where logic can get really difficult to implement in an asynchronous setting). There is little room to talk to the Akka guys about it as they are very convinced about what they are doing. I tried to on their mailing list…

16

@jstrachan

kotlin has that today using the ‘val’ keyword for values…

It seems to be not “complete immutability”:

class Man (val name: String, val age: Int) {
    fun renameTo(name: String){
        this.name = name
    }
}
val vasya = Man("Vasya", 20)
vasya.renameTo("Vasya Pupkin")
println(vasya.name)

will print:

Vasya Pupkin

17

You cannot reassign value to a read-only property, this line won’t compile.

18

@ilya.gorbunov
This code compiled and worked in Intellij Idea 2016.2 Kotlin REPL. Try it yoursel :wink:

19

This is a known bug in the REPL: https://youtrack.jetbrains.com/issue/KT-8064

The compiler should indeed reject this code during the compilation of source files.

20

Got it, thanks!