[英]Process communication of Python's Multiprocessing
I've learned about Python multiprocess's Pipes/Queues/Shared ctypes Objects/Managers, and I want to compare them with Linux's anonymous pipes, named pipes, shared memory, socket, and so on. 我已经了解了Python多进程的Pipes / Queues / Shared ctypes Objects / Managers,并且想将它们与Linux的匿名管道,命名管道,共享内存,套接字等进行比较。 I now have the following questions 我现在有以下问题
The pipes and queue modules of Python's multiprocessing are based on anonymous pipes. Python的多处理的管道和队列模块基于匿名管道。 Does it provide named pipes? 它提供命名管道吗?
Does Python multiprocessing.sharedctypes support independent process communication? Python multiprocessing.sharedctypes是否支持独立的进程通信? I think it only supports father and child process or brotherly process communication. 我认为它仅支持父子进程或兄弟进程通信。
Which of them are only used in the process of paternity or brotherhood, which can be communicated between independent processes or different hosts? 其中哪些仅用于亲子关系或兄弟情谊中,可以在独立过程或不同主机之间进行通信?
What are their respective characteristics, how should I choose them? 它们各自的特点是什么,我应该如何选择它们?
Thanks in advance. 提前致谢。
Your question is quite broad and most of the answers can be found in the multiprocessing
module documentation. 您的问题涉及面很广,大多数答案都可以在multiprocessing
模块文档中找到。
Here follows a somewhat short answer. 以下是一个简短的答案。
From the documentation : 从文档中 :
The multiprocessing.sharedctypes module provides functions for allocating ctypes objects from shared memory which can be inherited by child processes. multiprocessing.sharedctypes模块提供了用于从共享内存中分配ctypes对象的功能,这些对象可以由子进程继承。
You cannot use multiprocessing.sharedctypes
functionalities across processes which don't have parent/child relationship. 您不能跨没有父子关系的进程使用multiprocessing.sharedctypes
功能。
AF_INET
socket family can be used across different hosts. AF_INET
套接字系列可在不同主机上使用。 Nevertheless I'd recommend against it. 尽管如此,我还是建议不要这样做。 Rather use network sockets or some other abstraction mechanism. 而是使用网络套接字或其他某种抽象机制。 Python multiprocessing
module was initially implemented over the threading
APIs. Python multiprocessing
模块最初是通过threading
API实现的。 By the time, it grew in features it supports but the core idea remains the same. 到那时,它支持的功能有所增长,但核心思想保持不变。 The multiprocessing
module is intended to deal with Python process families. multiprocessing
模块旨在处理Python流程系列。 For any other use, the subprocess
module is a better option. 对于其他任何用途, subprocess
模块都是更好的选择。
For distribution of tasks and jobs across multiple hosts, there are far better solutions abstracting the low level infrastructure. 为了在多个主机之间分配任务和作业,有更好的解决方案抽象了低层基础结构。 You can take a look at Python projects such as Celery or Luigi or more complex infrastructures such as Apache Mesos . 您可以查看诸如Celery或Luigi之类的Python项目,或诸如Apache Mesos之类的更复杂的基础结构。
声明:本站的技术帖子网页,遵循CC BY-SA 4.0协议,如果您需要转载,请注明本站网址或者原文地址。任何问题请咨询:yoyou2525@163.com.