[英]Why doesn't the stream continue if the error occurs inside the Source
I have written a scala stream application. 我已经编写了一个scala流应用程序。 My objective is that if during the stream processing an item encounters an error during processing, then the stream ignores it and continues processing the remaining items.
我的目标是,如果在流处理过程中某个项目在处理过程中遇到错误,那么该流将忽略它并继续处理其余项目。
Towards this goal, I wrote this code 为了实现这个目标,我编写了这段代码
object StreamRestart extends App {
implicit val actorSystem = ActorSystem()
implicit val ec : ExecutionContext = actorSystem.dispatcher
var failed: ListBuffer[Int] = ListBuffer()
val decider : Supervision.Decider = {
case x : Exception => {
println(s"failed with error $x")
failed += x.getMessage.toInt
Supervision.Restart
}
}
implicit val actorMaterializer = ActorMaterializer(ActorMaterializerSettings(actorSystem).withSupervisionStrategy(decider))
def source : Source[Int, NotUsed] = {
val iter = (1 to 1000 toStream) map {x => if (x == 600) throw new Exception(x.toString) else x}
Source(iter)
}
val sink : Sink[Int, Future[Done]] = Sink.foreach[Int]{i => println(i)}
val future : Future[Done] = source.runWith(sink)
val f = future.map{_ =>
println("completed")
actorSystem.terminate()
}
Await.result(f, Duration.Inf)
println(s"who failed ${failed.toList}")
}
The above code crashes at 600 even though my decider says "Restart". 即使我的决定者说“重新启动”,上述代码也会在600崩溃。 If I move this exception side a "flow" then the decider works and the stream processes till it reaches 1000. but if the error occurs inside the source function, the application crashes.
如果我将此异常移到“流”侧,则决策程序将起作用,并且流将进行处理,直到达到1000。但是,如果源函数内部发生错误,则应用程序将崩溃。
Is there a way to make my application foolproof in a way that it always reaches the end. 有没有一种方法可以使我的应用程序万无一失,使它始终到最后。 Otherwise how do we recover from errors when it occurs in the source function.
否则,当源函数中发生错误时,我们如何从错误中恢复。
An error in the source is meant to be an unrecoverable failure on that source instance, which means the stream cannot continue using that source but you need to switch to a different source: it makes sense that if you are listening from a websocket and the http server goes down, you will not be able to listen again on that websocket. 源中的错误意味着该源实例上的不可恢复的故障,这意味着流无法继续使用该源,但是您需要切换到其他源:有意义的是,如果您正在从Websocket和http侦听服务器关闭,您将无法在该websocket上再次收听。
Akka streams offers recoverWithRetries
as described in the documentation to move to another source, or more generically to replace a part of the stream with a stream element with the same shape. Akka流按文档中所述提供
recoverWithRetries
,以移至另一个源,或更笼统地说,用具有相同形状的流元素替换流的一部分。
I would strongly recommend reading the documentation about handling errors in akka streams . 我强烈建议阅读有关处理akka流中的错误的文档。 To recover from failures you can use the operators called 'recover' or things like the 'Restart*' sources/sinks/flows.
要从故障中恢复,您可以使用称为“恢复”的运算符或类似“重新启动*”的源/接收器/流之类的东西。
The restart stages are most general and also most powerful really, so give them a look. 重新启动阶段是最通用的,实际上也是最强大的,因此请看一下它们。 Supervision also exists, which automatically handles such things but it has to be supported by a given operator.
监督也存在,它可以自动处理这些事情,但是必须由给定的操作员来支持。 (Most built in operators do, but not all, check their docs)
(大多数内置运算符可以(但不是全部)检查其文档)
声明:本站的技术帖子网页,遵循CC BY-SA 4.0协议,如果您需要转载,请注明本站网址或者原文地址。任何问题请咨询:yoyou2525@163.com.