[英]How to detect multiprocessing.Pipe is full?
问题描述:我正在Python中进行多处理,并使用multiprocessing.Pipe()在进程之间进行通信。 我已经进行了很多搜索,但是仍然找不到方法来检测管道是否已满。 例如,在下面的示例中,writePipe进程将数字继续放入2个不同的管道(奇数和偶数)中,而readPipe进程从这2个管道中连续读取数据。 但是,从奇数管道读取的速度要快得多,因此偶数管道将充满。 届时,writePipe进程将被阻塞,而readPipe进程仍继续等待从奇数管道读取数据,这将导致死锁。
我的问题:有什么方法可以检测到管道已满,以便在仍运行时停止将数字放入已满的管道中,而将数字放入仍然有空格的管道中?
from multiprocessing import Process, Pipe
def writePipe(sendNumberOdd, sendNumberEven):
i = 0
while True:
if i % 2 == 0:
sendNumberEven.send(i)
else:
sendNumberOdd.send(i)
i += 1
def readPipe(recvNumberOdd, recvNumberEven):
countEven = 0
while True:
countEven += 1
print(countEven, recvNumberEven.recv())
countOdd = 0
while countOdd < 50:
countOdd += 1
print (countOdd, recvNumberOdd.recv())
if __name__ == '__main__':
recvNumberOdd, sendNumberOdd = Pipe(duplex=False)
recvNumberEven, sendNumberEven = Pipe(duplex=False)
write = Process(target=writePipe, args=(sendNumberOdd, sendNumberEven))
read = Process(target=readPipe, args=(recvNumberOdd, recvNumberEven))
write.start()
read.start()
sendNumberOdd.close()
sendNumberEven.close()
提案未完成
class Connection(multiprocessing.Connection): def __init__(self, maxsize=0): self.__maxsize = maxsize self.size = 0 self.__lock = multiprocessing.Lock def send(self, obj): with self.__lock: self.size += sizeof(obj) super().send(obj) def recv(self): _recv = super().recv() with self.__lock: self.size -= sizeof(_recv) return _recv def full(self): if self.__maxsize > 0: return self.size >= self.__maxsize return None def Pipe(maxsize=0, duplex=True): return Connection(maxsize), Connection(maxsize)
实现poll()
以检查是否有任何可用的数据。
Python»文档: poll([timeout])
Return whether there is any data available to be read.
例如:
if recvNumberEven.poll():
countEven += 1
print(countEven, recvNumberEven.recv())
两者的替代使用wait(...)
multiprocessing.connection.wait(object_list,timeout = None)
Wait till an object in object_list is ready. Returns the list of those objects in object_list which are ready.
您可以使用select
模块中的select
函数来执行输出管道是否已满的测试。
import select
import multiprocessing.connection as mpc
def pipe_full(conn):
r, w, x = select.select([], [conn], [], 0.0)
return 0 == len(w)
i, o = mpc.Pipe(duplex=False)
n = 0
while not pipe_full(o):
o.send(n)
n += 1
print('{} items fit.'.format(n))
声明:本站的技术帖子网页,遵循CC BY-SA 4.0协议,如果您需要转载,请注明本站网址或者原文地址。任何问题请咨询:yoyou2525@163.com.