[英]Python multiprocessing Pool Queues communication
I'm trying to implement a pool of two processes that run in parallel and communicate through a queue. 我正在尝试实现一个并行运行并通过队列进行通信的两个进程池。
The goal is to have a writer process that passes a message to a reader process by using a queue . 我们的目标是具有通过使用队列传递消息到读取器处理的写入处理。
Each process is printing a feedback on the terminal in order to have a feedback. 每个过程都在终端上打印反馈以获得反馈。
Here is the code: 这是代码:
#!/usr/bin/env python
import os
import time
import multiprocessing as mp
import Queue
def writer(queue):
pid = os.getpid()
for i in range(1,4):
msg = i
print "### writer ", pid, " -> ", msg
queue.put(msg)
time.sleep(1)
msg = 'Done'
print '### '+msg
queue.put(msg)
def reader(queue):
pid = os.getpid()
time.sleep(0.5)
while True:
print "--- reader ", pid, " -> ",
msg = queue.get()
print msg
if msg == 'Done':
break
if __name__ == "__main__":
print "Initialize the experiment PID: ", os.getpid()
mp.freeze_support()
queue = mp.Queue()
pool = mp.Pool()
pool.apply_async(writer, (queue))
pool.apply_async(reader, (queue))
pool.close()
pool.join()
The output I am expecting should be something like this: 我期待的输出应该是这样的:
Initialize the experiment PID: 2341
writer 2342 -> 1
reader 2343 -> 1
writer 2342 -> 2
reader 2343 -> 2
writer 2342 -> 3
reader 2343 -> 3
Done
However I only get the line: 但是我只得到了这条线:
Initialize the experiment PID: 2341
then the script quits. 然后脚本退出。
What is the correct way to implement the interprocess communication of two processes in a pool that communicates through a queue? 在通过队列进行通信的池中实现两个进程的进程间通信的正确方法是什么?
I Used mp.Manager().Queue()
as the queue because we couldn't directly pass Queue
. 我使用mp.Manager().Queue()
作为队列,因为我们无法直接传递Queue
。 Trying to directly use the Queue
was causing exceptions but getting unhandled since we were using apply_async
. 尝试直接使用Queue
导致异常但由于我们使用apply_async
。
I updated your codes to: 我将您的代码更新为:
#!/usr/bin/env python
import os
import time
import multiprocessing as mp
import Queue
def writer(queue):
pid = os.getpid()
for i in range(1,4):
msg = i
print "### writer ", pid, " -> ", msg
queue.put(msg)
time.sleep(1)
msg = 'Done'
print '### '+msg
queue.put(msg)
def reader(queue):
pid = os.getpid()
time.sleep(0.5)
while True:
print "--- reader ", pid, " -> ",
msg = queue.get()
print msg
if msg == 'Done':
break
if __name__ == "__main__":
print "Initialize the experiment PID: ", os.getpid()
manager = mp.Manager()
queue = manager.Queue()
pool = mp.Pool()
pool.apply_async(writer, (queue,))
pool.apply_async(reader, (queue,))
pool.close()
pool.join()
And I got this output: 我得到了这个输出:
Initialize the experiment PID: 46182
### writer 46210 -> 1
--- reader 46211 -> 1
### writer 46210 -> 2
--- reader 46211 -> 2
### writer 46210 -> 3
--- reader 46211 -> 3
### Done
--- reader 46211 -> Done
I believe this is what you expected. 我相信这是你的期望。
声明:本站的技术帖子网页,遵循CC BY-SA 4.0协议,如果您需要转载,请注明本站网址或者原文地址。任何问题请咨询:yoyou2525@163.com.