[英]Python multiprocessing Pool Queues communication
我正在嘗試實現一個並行運行並通過隊列進行通信的兩個進程池。
我們的目標是具有通過使用隊列傳遞消息到讀取器處理的寫入處理。
每個過程都在終端上打印反饋以獲得反饋。
這是代碼:
#!/usr/bin/env python
import os
import time
import multiprocessing as mp
import Queue
def writer(queue):
pid = os.getpid()
for i in range(1,4):
msg = i
print "### writer ", pid, " -> ", msg
queue.put(msg)
time.sleep(1)
msg = 'Done'
print '### '+msg
queue.put(msg)
def reader(queue):
pid = os.getpid()
time.sleep(0.5)
while True:
print "--- reader ", pid, " -> ",
msg = queue.get()
print msg
if msg == 'Done':
break
if __name__ == "__main__":
print "Initialize the experiment PID: ", os.getpid()
mp.freeze_support()
queue = mp.Queue()
pool = mp.Pool()
pool.apply_async(writer, (queue))
pool.apply_async(reader, (queue))
pool.close()
pool.join()
我期待的輸出應該是這樣的:
Initialize the experiment PID: 2341
writer 2342 -> 1
reader 2343 -> 1
writer 2342 -> 2
reader 2343 -> 2
writer 2342 -> 3
reader 2343 -> 3
Done
但是我只得到了這條線:
Initialize the experiment PID: 2341
然后腳本退出。
在通過隊列進行通信的池中實現兩個進程的進程間通信的正確方法是什么?
我使用mp.Manager().Queue()
作為隊列,因為我們無法直接傳遞Queue
。 嘗試直接使用Queue
導致異常但由於我們使用apply_async
。
我將您的代碼更新為:
#!/usr/bin/env python
import os
import time
import multiprocessing as mp
import Queue
def writer(queue):
pid = os.getpid()
for i in range(1,4):
msg = i
print "### writer ", pid, " -> ", msg
queue.put(msg)
time.sleep(1)
msg = 'Done'
print '### '+msg
queue.put(msg)
def reader(queue):
pid = os.getpid()
time.sleep(0.5)
while True:
print "--- reader ", pid, " -> ",
msg = queue.get()
print msg
if msg == 'Done':
break
if __name__ == "__main__":
print "Initialize the experiment PID: ", os.getpid()
manager = mp.Manager()
queue = manager.Queue()
pool = mp.Pool()
pool.apply_async(writer, (queue,))
pool.apply_async(reader, (queue,))
pool.close()
pool.join()
我得到了這個輸出:
Initialize the experiment PID: 46182
### writer 46210 -> 1
--- reader 46211 -> 1
### writer 46210 -> 2
--- reader 46211 -> 2
### writer 46210 -> 3
--- reader 46211 -> 3
### Done
--- reader 46211 -> Done
我相信這是你的期望。
聲明:本站的技術帖子網頁,遵循CC BY-SA 4.0協議,如果您需要轉載,請注明本站網址或者原文地址。任何問題請咨詢:yoyou2525@163.com.