[英]why asyncio.Queue could not work as expected?
I am writing simple producer/consumer program. 我正在编写简单的生产者/消费者计划。
import zmq
@asyncio.coroutine
def receive_data(future,s):
print("begin to recv sth from.....socket"
my_data = s.recv()
future.set_result(my_data)
@asyncio.coroutine
def producer(loop,q,s):
while True:
future = asyncio.Future()
yield from receive_data(future,s)
data = str(future.result())
yield from q.put(data)
@asyncio.coroutine
def consumer(loop,q):
while True:
a = yield from q.get()
print("i am get..."+str(a)+"..."+str(type(a)))
loop = asyncio.get_event_loop()
c = zmq.Context()
s = c.socket(zmq.REP)
s.bind('tcp://127.0.0.1:5515')
q = asyncio.Queue()
tasks=[asyncio.Task(producer(loop,q,s)),asyncio.Task(comsumer(loop,q))]
loop.run_until_complete(asyncio.gather(*tasks))
loop.close()
s.close()
It appears the consumer has no chance to execute. 消费者似乎没有机会执行。
The sockets receive data every 500ms, so when yield from
in receive_data function suspends the producer coroutine, the consumer coroutine will print info. 套接字每500ms接收一次数据,因此当receive_data函数中的
yield from
暂停生产者协程时,消费者协程将打印信息。
What could explain this? 有什么可以解释这个?
s.recv()
is blocking call, so receive_data
hungs until new ZMQ message arrives. s.recv()
是阻塞调用,因此receive_data
挂起,直到新的ZMQ消息到达。
That blocks event loop and consumer has no chance to execute itself. 这阻止了事件循环,消费者没有机会自己执行。
You can pass zmq.NOBLOCK
flag to .recv
and call asyncio.sleep(0)
if no data available to give eventloop a chance to iterate over other ready tasks. 您可以将
zmq.NOBLOCK
标志传递给.recv
并调用asyncio.sleep(0)
如果没有可用的数据可以让eventloop有机会迭代其他就绪任务。
You are mixing synchronous and asynchronous calls, the results will be synchronous. 您正在混合同步和异步调用,结果将是同步的。 If you want to keep using asyncio, you should define an asynchronous context
c = zmq.asyncio.context()
and use ROUTER socket s = c.socket(zmq.ROUTER)
. 如果要继续使用asyncio,则应定义异步上下文
c = zmq.asyncio.context()
并使用ROUTER套接字s = c.socket(zmq.ROUTER)
。 Then, following asyncio syntax, you should yield from recv_multipart()
so your my_data = s.recv()
becomes my_data = yield from s.recv_multipart()
. 然后,遵循asyncio语法,你应该从
recv_multipart()
得到你的my_data = s.recv()
my_data = yield from s.recv_multipart()
变成my_data = yield from s.recv_multipart()
。
Here's a guideline of what needs to happen: 以下是需要发生的事情的指导原则:
Context
and ZMQEventLoop
from zmq.asyncio
ZMQEventLoop
Context
和zmq.asyncio
asyncio
use the zmq loop by calling set_event_loop()
set_event_loop()
使asyncio
使用zmq循环 Working example: 工作范例:
import asyncio
import zmq
from zmq.asyncio import Context, ZMQEventLoop
async def receive_data(s):
data = await s.recv()
print('receive_data', data)
return data
async def producer(q, s):
while True:
data = await receive_data(s)
await q.put(data)
async def consumer(q):
while True:
a = await q.get()
print('i got... {} ... {}'.format(a, type(a)))
loop = ZMQEventLoop()
asyncio.set_event_loop(loop)
c = Context()
s = c.socket(zmq.ROUTER)
s.setsockopt(zmq.ROUTER_RAW, 1)
s.bind('tcp://127.0.0.1:5515')
q = asyncio.Queue()
tasks=[producer(q, s), consumer(q)]
loop.run_until_complete(asyncio.gather(*tasks))
loop.close()
s.close()
You can test it out with ROUTER_RAW using telnet: 您可以使用telnet使用ROUTER_RAW测试它:
$ telnet localhost 5515
Trying ::1...
telnet: connect to address ::1: Connection refused
Trying 127.0.0.1...
Connected to localhost.
Escape character is '^]'.
test
abcd1234
^]
telnet> Connection closed.
$
The response from the app for this would be: 应用程序对此的响应是:
receive_data b'\x00\xa3\x8e\x1f)'
receive_data b''
i got... b'\x00\xa3\x8e\x1f)' ... <class 'bytes'>
i got... b'' ... <class 'bytes'>
receive_data b'\x00\xa3\x8e\x1f)'
receive_data b'test\r\n'
i got... b'\x00\xa3\x8e\x1f)' ... <class 'bytes'>
i got... b'test\r\n' ... <class 'bytes'>
receive_data b'\x00\xa3\x8e\x1f)'
receive_data b'abcd1234\r\n'
i got... b'\x00\xa3\x8e\x1f)' ... <class 'bytes'>
i got... b'abcd1234\r\n' ... <class 'bytes'>
receive_data b'\x00\xa3\x8e\x1f)'
receive_data b''
i got... b'\x00\xa3\x8e\x1f)' ... <class 'bytes'>
i got... b'' ... <class 'bytes'>
声明:本站的技术帖子网页,遵循CC BY-SA 4.0协议,如果您需要转载,请注明本站网址或者原文地址。任何问题请咨询:yoyou2525@163.com.