Creating a distributed memory service in Scala

Question

How do you think Scala can help us to write a distributed memory service like Redis (but multi-thread)?

Scala programming encourages us to use immutable data in order to avoid shared data and race conditions, however in this kind of data storage immutable data can't help us (I think).

Blocking threads would be the solution?

Answer 1

Scala deals with immutable data and mutable data equally well.

Sure it's highly encouraged to use immutable data - for any data which can be shared across threads/actors.

But it's also highly encouraged to use mutable data dedicated to individual threads/actors - ie in a context which can never be shared across threads/actors.

A simple high-level concurrency pattern is:

1. Make Mutable Data Private To Each Actor :
   • dedicate mutable types to individual actors
   • manipulate mutable data with your actor act/recieve/react methods
2. Only Share Immutable Data Between Actor s
   • send messages to other actors - but convert the mutable data to immutable data and only send the latter between actors.
3. Never Send a reference to an Actor in a message to Another Actor
   • Due to (1), Actors themselves become mutable
   • Due to (2), they shouldn't be shared
   • This stops ActorA from invoking any methods on ActorB - all communication should be via messages (methods ! and !! )
4. Identify long-running operations and make them asynchronous. Even simple operations that go beyond the CPU + L1 Cache + L2 Cache can can be 100,000s to 10,000,000s times slower than faster in-CPU operations, Eg reading volumous data from memory, reading/writing to disk, reading/writing to the network - see table at end: http://norvig.com/21-days.html
5. As a rule, don't block on asynchronous operations - it usually "takes out" a thread (consumes resources), if not carefully designed can lead to deadlock (runtime failure) and can always be handled in a non-blocking manner. If you need to process an asynchronous response, have the asynchronous request return a Future[T] as part of the initiation. Then specify response handling code via Future.onComplete (or even better via Future.flatMap/Map/withFilter/Recover and their for -comprehension equivalents - which are handy 'monadic'-wrappers around onComplete ).