[英]ZMQ missing events being propagated in jeromq scala
I am new to ZeroMQ and seem to be losing messages in a loop in my begin()
method. 我是ZeroMQ的新手,似乎在begin()
方法的循环中丢失了消息。
I'm wondering if I am missing a piece where I am not queuing messages or something? 我想知道我是否错过了我未在其中排队消息的东西?
When I cause an event on my publisher, that sends two messages to my subscriber with a small gap in between, I seem not to be getting the second message that is relayed. 当我在发布者上引发一个事件时,该事件将两条消息发送给我的订阅者,并且两者之间的间隔很小,我似乎并没有收到要中继的第二条消息。 What am I missing? 我想念什么?
class ZMQSubscriber[T <: Transaction, B <: Block](
socket: InetSocketAddress,
hashTxListener: Option[HashDigest => Future[Unit]],
hashBlockListener: Option[HashDigest => Future[Unit]],
rawTxListener: Option[Transaction => Future[Unit]],
rawBlockListener: Option[Block => Future[Unit]]) {
private val logger = BitcoinSLogger.logger
def begin()(implicit ec: ExecutionContext) = {
val context = ZMQ.context(1)
// First, connect our subscriber socket
val subscriber = context.socket(ZMQ.SUB)
val uri = socket.getHostString + ":" + socket.getPort
//subscribe to the appropriate feed
hashTxListener.map { _ =>
subscriber.subscribe(HashTx.topic.getBytes(ZMQ.CHARSET))
logger.debug("subscribed to the transaction hashes from zmq")
}
rawTxListener.map { _ =>
subscriber.subscribe(RawTx.topic.getBytes(ZMQ.CHARSET))
logger.debug("subscribed to raw transactions from zmq")
}
hashBlockListener.map { _ =>
subscriber.subscribe(HashBlock.topic.getBytes(ZMQ.CHARSET))
logger.debug("subscribed to the hashblock stream from zmq")
}
rawBlockListener.map { _ =>
subscriber.subscribe(RawBlock.topic.getBytes(ZMQ.CHARSET))
logger.debug("subscribed to raw block")
}
subscriber.connect(uri)
subscriber.setRcvHWM(0)
logger.info("Connection to zmq client successful")
while (true) {
val notificationTypeStr = subscriber.recvStr(ZMQ.DONTWAIT)
val body = subscriber.recv(ZMQ.DONTWAIT)
Future(processMsg(notificationTypeStr, body))
}
}
private def processMsg(topic: String, body: Seq[Byte])(implicit ec: ExecutionContext): Future[Unit] = Future {
val notification = ZMQNotification.fromString(topic)
val res: Option[Future[Unit]] = notification.flatMap {
case HashTx =>
hashTxListener.map { f =>
val hash = Future(DoubleSha256Digest.fromBytes(body))
hash.flatMap(f(_))
}
case RawTx =>
rawTxListener.map { f =>
val tx = Future(Transaction.fromBytes(body))
tx.flatMap(f(_))
}
case HashBlock =>
hashBlockListener.map { f =>
val hash = Future(DoubleSha256Digest.fromBytes(body))
hash.flatMap(f(_))
}
case RawBlock =>
rawBlockListener.map { f =>
val block = Future(Block.fromBytes(body))
block.flatMap(f(_))
}
}
}
}
So this seems to have been solved by using a ZMsg.recvMsg()
in the while
-loop instead of 因此,这似乎已经通过在while
-loop中使用ZMsg.recvMsg()
解决了,
val notificationTypeStr = subscriber.recvStr(ZMQ.DONTWAIT)
val body = subscriber.recv(ZMQ.DONTWAIT)
I'm not sure why this works, but it does. 我不确定为什么会这样,但是确实可以。 So here is what my begin
method looks like now 所以这是我的begin
方法现在的样子
while (run) {
val zmsg = ZMsg.recvMsg(subscriber)
val notificationTypeStr = zmsg.pop().getString(ZMQ.CHARSET)
val body = zmsg.pop().getData
Future(processMsg(notificationTypeStr, body))
}
Future.successful(Unit)
}
What am I missing? 我想念什么?
The trick is in the (non-)blocking mode of the respective call to the .recv()
method. 诀窍在于对.recv()
方法的相应调用的(非)阻止模式 。
A second call to the subscriber.recv( ZMQ.DONTWAIT )
-method thus returns immediately, so your second part, ( the body
) may and will legally contain nothing, even though your promise stated a pair of messages was indeed dispached from the publisher-side ( a pair of .send()
method calls - one may also object, there are chances the sender was actually sending just one message, in a multi-part fashion - MCVE-code is not specific on this part ). 因此,第二次调用subscriber.recv( ZMQ.DONTWAIT )
方法将立即返回,因此即使您的承诺表明确实有一对消息已从发布subscriber.recv( ZMQ.DONTWAIT )
,您的第二部分( body
)也可能并且将不包含任何内容。侧(一对.send()
方法调用-一个也可能是对象,发送者有可能实际上只是以多部分方式发送一条消息-MCVE代码在这一部分上不是特定的)。
So, once you have moved your code from non-blocking mode ( in the O/P ) into a principally blocking-mode ( which locked / sync-ed the further flow of the code with the external event of an arrival of any plausibly formatted message, not returning earlier ), in: 因此,一旦您将代码从非阻塞模式(在O / P中)移到了一个主要的阻塞模式(该模式将锁定/同步代码的进一步流程以及任何可能被格式化的外部事件的发生,消息,不早返回),位于:
val zmsg = ZMsg.recvMsg(subscriber) // which BLOCKS-till-a-1st-zmsg-arrived
both the further processed .pop()
-ed parts just unload the components ( ref. the remark on actual ZMsg
multi-part structure actually sent by the published-side, presented above ) 这两个经过进一步处理的.pop()
ed部分都只是卸载了组件(请ZMsg
上面由发布端实际发送的有关实际ZMsg
多部分结构的说明)
the code surprised me on several points. 该代码使我感到惊讶。 Besides a rather very "late" call to the .connect()
-method, compared to all the previous socket-archetype detailed settings ( that normally get arranged "after" a request to setup a connection ). 与之前所有的套接字原型详细设置(通常在“建立连接的请求之后”安排.connect()
相比,除了对.connect()
方法的非常“晚”的调用之外。 While this may work fine, as intended, yet it exposes even tighter ( smaller ) time-window for the .Context()
-instance to setup and (re-)negotiate all the relevant connection-details so as to become RTO. 尽管这可能按预期工作得很好,但它为.Context()
实例提供了更紧凑的(较小的)时间窗口来设置和(重新)协商所有相关的连接细节,从而成为RTO。
One particular line attracted my attention: subscriber.setRcvHWM( 0 )
this is a version-archetype dependent trick. 其中一行引起了我的注意: subscriber.setRcvHWM( 0 )
这是一个依赖于版本原型的技巧。 Yet, the value of zero causes an application to become vulnerable and I would not advise doing so in any production-grade application. 但是,零值会导致应用程序易受攻击,我不建议在任何生产级应用程序中这样做。
声明:本站的技术帖子网页,遵循CC BY-SA 4.0协议,如果您需要转载,请注明本站网址或者原文地址。任何问题请咨询:yoyou2525@163.com.