[英]ZMQ devices, ioloop and multiprocessing
I am trying to apply the PUSH/PULL pattern as depicts in the following figure: 我正在尝试应用下图所示的PUSH / PULL模式:
| PULL ---> Send via HTTP
| PULL ---> Send via HTTP
---- PUSH ----- DEVICE ---- | PULL ---> Send via HTTP
| PULL ---> Send via HTTP
| PULL ---> Send via HTTP
The PUSH
socket connects to the ZeroMQ device and emits the messages which are then propagated to all connected PULL
sockets. PUSH
套接字连接到ZeroMQ设备,并发出消息,然后消息传播到所有连接的PULL
套接字。 What I want to achieve is a kind of parallel processing over multiple nodes in a pipeline. 我要实现的是一种对管道中多个节点的并行处理。 When processing has been done by the
PULL
socket, it should forward the message via HTTP to the remote endpoint. 当通过
PULL
套接字完成处理后,它应该通过HTTP将消息转发到远程端点。
Here is the code: 这是代码:
from multiprocessing import Process
import random
import time
import zmq
from zmq.devices import ProcessDevice
from zmq.eventloop import ioloop
from zmq.eventloop.zmqstream import ZMQStream
ioloop.install()
bind_in_port = 5559
bind_out_port = 5560
dev = ProcessDevice(zmq.STREAMER, zmq.PULL, zmq.PUSH)
dev.bind_in("tcp://127.0.0.1:%d" % bind_in_port)
dev.bind_out("tcp://127.0.0.1:%d" % bind_out_port)
dev.setsockopt_in(zmq.IDENTITY, b'PULL')
dev.setsockopt_out(zmq.IDENTITY, b'PUSH')
dev.start()
time.sleep(2)
def push():
context = zmq.Context()
socket = context.socket(zmq.PUSH)
socket.connect("tcp://127.0.0.1:%s" % bind_in_port)
server_id = random.randrange(1,10005)
for i in range(5):
print("Message %d sent" % i)
socket.send_string("Push from %s" % server_id)
def pull():
context = zmq.Context()
socket = context.socket(zmq.PULL)
socket.connect("tcp://127.0.0.1:%s" % bind_out_port)
loop = ioloop.IOLoop.instance()
pull_stream = ZMQStream(socket, loop)
def on_recv(message):
print(message)
pull_stream.on_recv(on_recv)
loop.start()
Process(target=push).start()
time.sleep(2)
for i in range(2):
Process(target=pull).start()
Although the messages are correctly sent to the ZeroMQ device, I cannot see any message received - the on_recv
callback is never being called. 尽管消息已正确发送到ZeroMQ设备,但我看不到收到的任何消息-永远不会调用
on_recv
回调。
Any help is appreciated. 任何帮助表示赞赏。
Thanks 谢谢
there's missing code above in the device init to provide a full answer. 设备初始化中缺少上面的代码来提供完整的答案。 what are dev and *port ?
什么是dev和* port? 1 thing could be to add the sleep(1) after .connect for ports to stabilize note: no need to set identity on push/pull
一件事可能是在.connect之后添加sleep(1)以使端口稳定注意:无需在推/拉上设置标识
声明:本站的技术帖子网页,遵循CC BY-SA 4.0协议,如果您需要转载,请注明本站网址或者原文地址。任何问题请咨询:yoyou2525@163.com.