for-comprehension vs Future.sequence

Question

There are 4 cores on my machine.

And I thought that for-comprehension schedules stuff in paraller (not to be depended on previous result..same as flatMap):

val stuffResult: Future[String] = for {
   stuff1 <- service1.doServiceStuff("service 1")  // worker-5 (4+1)
   stuff2 <- service1.doServiceStuff("service 2")
   stuff3 <- service1.doServiceStuff("service 3")
   stuff4 <- service1.doServiceStuff("service 4")
 } yield (stuff1 + ", " + stuff2 + ", "+ stuff3 + ", " + stuff4)

where

 class Service {

 implicit val blockingExContext = scala.concurrent.ExecutionContext.fromExecutor(null: Executor)

 def doServiceStuff(name:String): Future[String] = {
    Future {
      blocking {    
        println ( s"start ${name} on " + Thread.currentThread.getName)
        Thread.sleep(5000)
        "stuff_done"
      }
    }
  }
}

But what i see is (each step takes ~5 seconds):

• start service 1 on ForkJoinPool-3-worker-5
• start service 2 on ForkJoinPool-3-worker-5
• start service 3 on ForkJoinPool-3-worker-5
• ERROR:java.util.concurrent.TimeoutException: Futures timed out after [10 seconds]

all runs on one thread, instead of use existing FREE one and finish all as fast as possible - in ~5 secs.

But if I go with:

val stuff1 = service1.doServiceStuff("service 1")
val stuff2 = service1.doServiceStuff("service 2")
val stuff3 = service1.doServiceStuff("service 3")
val stuff4 = service1.doServiceStuff("service 4")

Future.sequence(List(stuff1, stuff2, stuff3, stuff4)).map { list =>
  list.foldLeft("") { (acc, x) => acc + " " + x }
}
..

all ends in 5 second.

At which pont for-comprehension works sequentially? Does it?

Answer 1

It does not work sequentially, it just cannot start the Future s till they are created (which would happen in your case in your flatMap of your previous Future ), so you need to create them in advance if you want to process them in parallel (with the usual implicit ExecutionContext s).

Probably this tutorial explains better though (it complicates it with withFilter ):

The purchase future is completed only once both usdQuote and chfQuote are completed– it depends on the values of both these futures so its own computation cannot begin earlier.

The for-comprehension above is translated into:

val purchase = usdQuote flatMap { usd => chfQuote .withFilter(chf => isProfitable(usd, chf)) .map(chf => connection.buy(amount, chf)) }

which is a bit harder to grasp than the for-comprehension, but we analyze it to better understand the flatMap operation. The flatMap operation maps its own value into some other future. Once this different future is completed, the resulting future is completed with its value. In our example, flatMap uses the value of the usdQuote future to map the value of the chfQuote into a third future which sends a request to buy a certain amount of Swiss francs. The resulting future purchase is completed only once this third future returned from map completes.

What you would really only need is something like map2 instead of flatMap , as you do not use the returned value from the previous Future to create the new Future .