[英]What IPC mechanism is used to share the data in multiprocessing.Queue between two Python processes?
Here is my Python code. 这是我的Python代码。
#!/usr/bin/env python
import multiprocessing
import time
def worker(q):
while True:
data = q.get()
print 'worker: got: %d' % data
if data == -1:
print 'worker: done'
break
def master():
q = multiprocessing.Queue()
p = multiprocessing.Process(target=worker, args=(q,))
p.start()
for i in range(5):
q.put(i)
print 'master: put: %d' % i
time.sleep(1)
q.put(-1)
p.join()
print 'master: done'
master()
print 'exiting ...'
Here is the output when I run this code on a Debian 9 GNU/Linux system. 这是我在Debian 9 GNU / Linux系统上运行此代码时的输出。
$ python q.py
master: put: 0
worker: got: 0
master: put: 1
worker: got: 1
master: put: 2
worker: got: 2
master: put: 3
worker: got: 3
master: put: 4
worker: got: 4
worker: got: -1
worker: done
master: done
exiting ...
I am trying to find some evidence that the the master process and the worker process are communicating the data being put in the queue via a socket, message queue or shared memory. 我正在尝试寻找一些证据,证明主进程和工作进程正在通过套接字,消息队列或共享内存传递正在放入队列中的数据。 But I cannot seem to find any evidence in favour of it.
但是我似乎找不到任何证据支持它。
$ ps -ef | grep python | grep -v grep; netstat -nopa | grep python; ipcs
lone 2914 9836 2 12:54 pts/1 00:00:00 python q.py
lone 2915 2914 0 12:54 pts/1 00:00:00 python q.py
(Not all processes could be identified, non-owned process info
will not be shown, you would have to be root to see it all.)
------ Message Queues --------
key msqid owner perms used-bytes messages
------ Shared Memory Segments --------
key shmid owner perms bytes nattch status
0x00000000 163840 lone 600 393216 2 dest
0x00000000 262145 lone 600 393216 2 dest
0x00000000 360450 lone 600 393216 2 dest
0x00000000 142344195 lone 600 524288 2 dest
0x00000000 101941252 lone 600 4194304 2 dest
0x00000000 950277 lone 600 524288 2 dest
0x00000000 118194183 lone 600 696320 2 dest
0x00000000 118292488 lone 600 4153344 2 dest
0x00000000 117899273 lone 600 4153344 2 dest
0x00000000 118226954 lone 600 696320 2 dest
0x00000000 123535371 lone 600 86016 2 dest
0x00000000 123338764 lone 600 1728512 2 dest
0x00000000 123240461 lone 600 1798144 2 dest
0x00000000 123568144 lone 600 86016 2 dest
0x00000000 137330705 lone 600 32768 2 dest
0x00000000 137232402 lone 600 81920 2 dest
0x00000000 29098003 lone 600 4194304 2 dest
0x00000000 137265172 lone 600 81920 2 dest
0x00000000 137297941 lone 600 151552 2 dest
0x00000000 35258391 lone 600 393216 2 dest
0x00000000 35291160 lone 600 12288 2 dest
0x00000000 35323929 lone 600 12288 2 dest
0x00000000 35356698 lone 600 393216 2 dest
0x00000000 35520539 lone 600 12288 2 dest
0x00000000 35422236 lone 600 393216 2 dest
0x00000000 35455005 lone 600 12288 2 dest
------ Semaphore Arrays --------
key semid owner perms nsems
multiprocessing.Queue
transferred from the master process to the worker process? multiprocessing.Queue
从主进程转移到工作进程? multiprocessing.Queue
from the master process to the worker process? multiprocessing.Queue
中将数据从主进程传输到工作进程? Python is open source, and you can see the implementation plainly here: https://github.com/python/cpython/blob/master/Lib/multiprocessing/queues.py Python是开源的,您可以在这里简单地看到实现: https : //github.com/python/cpython/blob/master/Lib/multiprocessing/queues.py
It uses a Pipe
which is implemented here: https://github.com/python/cpython/blob/master/Lib/multiprocessing/connection.py 它使用在此处实现的
Pipe
: https : //github.com/python/cpython/blob/master/Lib/multiprocessing/connection.py
Which uses os.pipe()
, implemented here: https://github.com/python/cpython/blob/master/Modules/posixmodule.c 使用
os.pipe()
,在此处实现: https : //github.com/python/cpython/blob/master/Modules/posixmodule.c
Which uses pipe2()
, documented here: http://man7.org/linux/man-pages/man2/pipe.2.html 使用
pipe2()
文件,在此处记录: http : pipe2()
声明:本站的技术帖子网页,遵循CC BY-SA 4.0协议,如果您需要转载,请注明本站网址或者原文地址。任何问题请咨询:yoyou2525@163.com.