平衡和广播之间的差异扇出Akka Streams

Question

I have a little confusion with fan out strategies in Akka streams , I read that Broadcast – (1 input, N outputs) given an input element emits to each output, while Balance – (1 input, N outputs) given an input element emits to one of its output ports.

Can you explain me:

1. How balance does work with multiple consumers?
2. Meaning of phrase "emits to one of its output ports"
3. Does port is same to downstream?
4. Does 'Balance' stand for replication of input stream into a few output partition
5. What does "balance is enabling graphs to be split apart and multiple instances of downstream subscribers replicated to handle the volume" mean?

Answer 1

From the documentation... broadcast emits (sends) the element to every consumer. balance only emits to the first available consumer.

broadcast

Emit each incoming element each of n outputs.

balance

Fan-out the stream to several streams. Each upstream element is emitted to the first available downstream consumer.

EDIT from comments:

From your gist, you should make two averageCarrierDelay functions, one for each Z and F . Then you can see all the elements sent to each.

val averageCarrierDelayZ =
    Flow[FlightDelayRecord]
      .groupBy(30, _.uniqueCarrier)
        .fold(("", 0, 0)){
          (x: (String, Int, Int), y:FlightDelayRecord) => {
            println(s"Z Received Element: ${y}")
            val count = x._2 + 1
            val totalMins = x._3 + Try(y.arrDelayMins.toInt).getOrElse(0)
            (y.uniqueCarrier, count, totalMins)
          }
        }.mergeSubstreams


val averageCarrierDelayF =
    Flow[FlightDelayRecord]
      .groupBy(30, _.uniqueCarrier)
        .fold(("", 0, 0)){
          (x: (String, Int, Int), y:FlightDelayRecord) => {
            println(s"F Received Element: ${y}")
            val count = x._2 + 1
            val totalMins = x._3 + Try(y.arrDelayMins.toInt).getOrElse(0)
            (y.uniqueCarrier, count, totalMins)
          }
        }.mergeSubstreams

Edit 2: To check things in the future I'd recommend a generic logger for stream stages so you can see what is going on.

def logElement[A](msg: String) = Flow[A].map { a => println(s"${msg} ${a}"); a }

Doing this allows you to do something like:

D ~> logElement[FlightDelayRecord]("F received: ") ~> F
D ~> logElement[FlightDelayRecord]("Z received: ") ~> Z

This way you can check areas of your graph for strange behavior that you may or may not be expecting.

Answer 2

As others have already said, broadcast emits its input to all output ports, while balance emits its input to one output port based on backpressure.

When you use GraphStage , you need to choose which output port you want to use. Consider this example:

val q1 = Source.queue[Int](10, OverflowStrategy.fail)
val q2 = Source.queue[Int](10, OverflowStrategy.fail)
GraphDSL.create(q1, q2)(Keep.both) { implicit b => (input1, input2) =>
  import GraphDSL.Implicits._

  val broadcast = b.add(Broadcast[Int](2))
  val balance = b.add(Balance[Int](2))

  val consumer1, consumer2, consumer3, consumer4 = b.add(Sink.foreach[Int](println))

  input1 ~> broadcast.in
  input2 ~> balance.in

  broadcast.out(0) ~> consumer1
  broadcast.out(1) ~> consumer2

  balance.out(0) ~> consumer3
  balance.out(1) ~> consumer4

  ClosedShape
}

Here we connect one input to a broadcast stage and one to a balance stage. Then we connect different output ports of the broadcast and balance stages to the respective consumers.

In this particular case, when you run the stream, elements coming through the first input will be passed to both consumer1 and consumer2 , because a broadcast stage copies its input to all its outputs (and here are two outputs), and elements coming through the second input will be distributed evenly across consumer3 and consumer4 based on the speed of your terminal (ie the speed of println ), because Sink.foreach backpressures when its function executes for long time.

Note the we have specified that the broadcast and balance stages have 2 ports each (when calling their factory methods), and that we have specified which output port we connect to which consumer.