简体   繁体   English

如何在多处理中将模块用于“全局”变量?

[英]How to use a module for “global” variables with multiprocessing?

I am using the common pattern where a module has a number of module-level variables and is imported by other modules, eg module a, b, c... etc import module 'globalv' which has, for examples, module-level variable foo with value 'bar'. 我正在使用一种通用模式,其中一个模块具有多个模块级别的变量,并由其他模块导入,例如,模块a,b,c ...等导入模块'globalv',例如具有模块级别的变量值为'bar'的foo。 So far so good, works as expected in a single-process environment, and one can import the module and/or it's variables. 到目前为止,它可以在单进程环境中按预期工作,并且可以导入模块和/或其变量。 I like this pattern, it makes the code clean and easily readable, and IDEs such as eclipse auto-complete the variable names. 我喜欢这种模式,它使代码清晰易读,并且诸如eclipse之类的IDE会自动完成变量名。

However, in a multiprocessing environment this doesn't work as described in the python documentation because each process has a separate interpreter with distinct address spaces. 但是,在多处理环境中,这无法像python文档中所述那样工作,因为每个进程都有一个单独的解释器,这些解释器具有不同的地址空间。

Now, I am wondering if perhaps there is commonly accepted way to make this pattern work with multiprocessing ? 现在,我想知道是否存在使该模式与多处理一起工作的普遍接受的方法? Perhaps with Manager, Value, or Array ? 也许使用Manager,Value或Array?

My thoughts are to make globalv.foo a Value or Manager instance and pass it to the child processes as and argument; 我的想法是使globalv.foo成为Value或Manager实例,并将其作为and参数传递给子进程。 I only use simple, pickleable objects, eg integers, and strings for my "global" variables. 我只对“全局”变量使用简单的可腌制对象,例如整数和字符串。 The first thing to do when the child process runs is then to replace foo in the child process with the foo passed by the parent process, which is a Value or Manager instance. 然后,子进程运行时要做的第一件事就是用父进程传递的foo(值或Manager实例)替换子进程中的foo。 It should then be the same as the original foo in the parent process and updating foo in the parent or child process should be shared. 然后,它应该与父进程中的原始foo相同,并且应该共享父进程或子进程中的更新foo。 However, this seems clumsy, any ideas how to do this better be welcome. 但是,这似乎很笨拙,任何更好地做到这一点的想法都应受到欢迎。

You could make use of the shared memory functionality of multiprocessing: 您可以利用多重处理的共享内存功能:

https://docs.python.org/3.9/library/multiprocessing.shared_memory.html https://docs.python.org/3.9/library/multiprocessing.shared_memory.html

声明:本站的技术帖子网页,遵循CC BY-SA 4.0协议,如果您需要转载,请注明本站网址或者原文地址。任何问题请咨询:yoyou2525@163.com.

 
粤ICP备18138465号  © 2020-2024 STACKOOM.COM