[英]Akka streams: dealing with futures within graph stage
Within an akka stream stage FlowShape[A, B]
, part of the processing I need to do on the A's is to save/query a datastore with a query built with A data. 在akka流阶段
FlowShape[A, B]
,我需要对A执行的部分处理是使用使用A数据构建的查询来保存/查询数据存储。 But that datastore driver query gives me a future, and I am not sure how best to deal with it (my main question here). 但是那个数据存储驱动程序查询给了我一个未来,我不知道如何最好地处理它(我的主要问题在这里)。
case class Obj(a: String, b: Int, c: String)
case class Foo(myobject: Obj, name: String)
case class Bar(st: String)
//
class SaveAndGetId extends GraphStage[FlowShape[Foo, Bar]] {
val dao = new DbDao // some dao with an async driver
override def createLogic(inheritedAttributes: Attributes) = new GraphStageLogic(shape) {
setHandlers(in, out, new InHandler with Outhandler {
override def onPush() = {
val foo = grab(in)
val add = foo.record.value()
val result: Future[String] = dao.saveAndGetRecord(add.myobject)//saves and returns id as string
//the naive approach
val record = Await(result, Duration.inf)
push(out, Bar(record))// ***tests pass every time
//mapping the future approach
result.map { x=>
push(out, Bar(x))
} //***tests fail every time
The next stage depends on the id of the db record returned from query, but I want to avoid Await
. 下一个阶段取决于从查询返回的db记录的id,但我想避免
Await
。 I am not sure why mapping approach fails: 我不确定为什么映射方法失败:
"it should work" in {
val source = Source.single(Foo(Obj("hello", 1, "world")))
val probe = source
.via(new SaveAndGetId))
.runWith(TestSink.probe)
probe
.request(1)
.expectBarwithId("one")//say we know this will be
.expectComplete()
}
private implicit class RichTestProbe(probe: Probe[Bar]) {
def expectBarwithId(expected: String): Probe[Bar] =
probe.expectNextChainingPF{
case r @ Bar(str) if str == expected => r
}
}
When run with mapping future, I get failure: 当运行映射未来时,我会失败:
should work ***FAILED***
java.lang.AssertionError: assertion failed: expected: message matching partial function but got unexpected message OnComplete
at scala.Predef$.assert(Predef.scala:170)
at akka.testkit.TestKitBase$class.expectMsgPF(TestKit.scala:406)
at akka.testkit.TestKit.expectMsgPF(TestKit.scala:814)
at akka.stream.testkit.TestSubscriber$ManualProbe.expectEventPF(StreamTestKit.scala:570)
The async side channels example in the docs has the future in the constructor of the stage, as opposed to building the future within the stage, so doesn't seem to apply to my case. 文档中的异步侧通道示例在舞台的构造函数中具有未来,而不是在舞台中构建未来,因此似乎不适用于我的情况。
I agree with Ramon. 我同意拉蒙的观点。 Constructing a new
FlowShape
is not necessary in this case and it is too complicated. 在这种情况下,不需要构造新的
FlowShape
并且它太复杂了。 It is very much convenient to use mapAsync
method here: 这里使用
mapAsync
方法非常方便:
Here is a code snippet to utilize mapAsync
: 以下是使用
mapAsync
的代码段:
import akka.stream.scaladsl.{Sink, Source}
import scala.concurrent.ExecutionContext.Implicits.global
import scala.concurrent.Future
object MapAsyncExample {
val numOfParallelism: Int = 10
def main(args: Array[String]): Unit = {
Source.repeat(5)
.mapAsync(parallelism)(x => asyncSquare(x))
.runWith(Sink.foreach(println)) previous stage
}
//This method returns a Future
//You can replace this part with your database operations
def asyncSquare(value: Int): Future[Int] = Future {
value * value
}
}
In the snippet above, Source.repeat(5)
is a dummy source to emit 5
indefinitely. 在上面的代码片段中,
Source.repeat(5)
是一个无限发出5
的虚拟源。 There is a sample function asyncSquare
which takes an integer
and calculates its square in a Future
. 有一个示例函数
asyncSquare
,它接受一个integer
并在Future
计算它的平方。 .mapAsync(parallelism)(x => asyncSquare(x))
line uses that function and emits the output of Future
to the next stage. .mapAsync(parallelism)(x => asyncSquare(x))
行使用该函数并将Future
的输出发送到下一个阶段。 In this snipet, the next stage is a sink
which prints every item. 在这种snipet,下一阶段是
sink
其打印每一个项目。
parallelism
is the maximum number of asyncSquare
calls that can run concurrently. parallelism
是可以并发运行的最大asyncSquare
调用数。
I think your GraphStage
is unnecessarily overcomplicated. 我认为你的
GraphStage
不必要地过于复杂。 The below Flow
performs the same actions without the need to write a custom stage: 以下
Flow
执行相同的操作,无需编写自定义阶段:
val dao = new DbDao
val parallelism = 10 //number of parallel db queries
val SaveAndGetId : Flow[Foo, Bar, _] =
Flow[Foo]
.map(foo => foo.record.value().myobject)
.mapAsync(parallelism)(rec => dao.saveAndGetRecord(rec))
.map(Bar.apply)
I generally try to treat GraphStage
as a last resort, there is almost always an idiomatic way of getting the same Flow by using the methods provided by the akka-stream library. 我通常会尝试将
GraphStage
视为最后的手段,几乎总是通过使用akka-stream库提供的方法获得相同的Flow的惯用方法。
声明:本站的技术帖子网页,遵循CC BY-SA 4.0协议,如果您需要转载,请注明本站网址或者原文地址。任何问题请咨询:yoyou2525@163.com.