简体   繁体   English

运行时错误:多处理(python 3.9)

[英]runtime error: Multiprocessing (python 3.9)

I am trying to run the code available here https://github.com/GDPlumb/MAPLE/blob/master/1-Accuracy/run.py but the following error occurs.我正在尝试运行https://github.com/GDPlumb/MAPLE/blob/master/1-Accuracy/run.py此处可用的代码,但出现以下错误。 Any idea how can I solve it知道我该如何解决

child processes and you have forgotten to use the proper idiom                          

main主要的
in the在里面
module:模块:
if name main freeze=support() -- if name main freeze=support() --
"The ""freeze_support()"" line can be omitted if the program" """freeze_support()"" 行可以省略,如果程序"
"return Pool(processes, initializer, initargs, rnaxtasksperchild," “返回池(进程,初始化程序,initargs,rnaxtasksperchild,”
"File ""C:\Python39\lib\multiprocessing\pool.py"", line 212, in init" "文件 ""C:\Python39\lib\multiprocessing\pool.py"",第 212 行,在 init"
"File ""C:\Python39\lib\multiprocessing\pool.py"", line 303, return self._repopulate_pool_static(self._ctx, self.Pro" "文件 ""C:\Python39\lib\multiprocessing\pool.py"",第 303 行,返回 self._repopulate_pool_static(self._ctx, self.Pro"
"File ""C:\Python39\lib\multiprocessing\pool.py"", line 326," "文件 ""C:\Python39\lib\multiprocessing\pool.py"",第 326 行,"
w.start() w.start()
"""C:\Python39\lib\multiprocessing\process.py"", line 121, in start lf._popen = self._Popen(self)" """C:\Python39\lib\multiprocessing\process.py"",第 121 行,开始 lf._popen = self._Popen(self)"
_static _静止的
"in _repopulate_pool cess," “在 _repopulate_pool 进程中,”
in _repopulate_pool在 _repopulate_pool
File se文件服务
File re """C:\Python39\lib\multiproce ssing\context.py"", line 327, in turn Popen(process_obj)" _Popen "File ""C:\Python39\lib\multiproce ssing\popen_spawn_win32.py"", line 45,"文件 re """C:\Python39\lib\multiproce ssing\context.py"",第 327 行,依次 Popen(process_obj)" _Popen "文件 ""C:\Python39\lib\multiproce ssing\popen_spawn_win32.py" “,第 45 行,”

prep_data = spawn.get_preparation_data(process_obj._name) prep_data = spawn.get_preparation_data(process_obj._name)
"File ""c:\Python39\lib\multiprocessing\spawn.py"", line 154,in get_preparation_data" "文件 ""c:\Python39\lib\multiprocessing\spawn.py"",第 154 行,在 get_preparation_data 中"
_check_not_importing _check_not_importing
main ()主要的 ()
main主要的
"File ""C:\Python39\lib\multiprocessing\spawn.py"",line 134,in check_not_importing raise RuntimeError ('''" "文件 ""C:\Python39\lib\multiprocessing\spawn.py"",第 134 行,在check_not_importing中引发 RuntimeError ('''"
RuntimeError:运行时错误:
before
An attempt has been made to start a new process current process has finished its bootstrapping phase.已尝试启动一个新进程,当前进程已完成其引导阶段。
This probably means that you are not using fork to这可能意味着您没有使用 fork
the
your您的
start开始
child processes and you have forgotten to use the proper idiom in the main module:子进程并且您忘记在主模块中使用正确的习语:
if name main reeze_support() if name main reeeze_support()
"The ""freeze_support()"" line can be omitted if the program" """freeze_support()"" 行可以省略,如果程序"
is
"l.py""," "l.py"","
"File ""C:\Python39\lib\multiprocessing\poo" "文件 ""C:\Python39\lib\multiprocessing\poo"
"line 326, in _repopulate_pool_static" “第 326 行,在 _repopulate_pool_static 中”
"File ""C:\Python39\lib\multiprocessing\process.py"", line 121, in start" "文件 ""C:\Python39\lib\multiprocessing\process.py"",第 121 行,开始"
"File ""C:\Python39\lib\multiproce ssin \context. "" line 327, in Popen" "文件 ""C:\Python39\lib\multiproce ssin \context."" 第 327 行,在 Popen 中"

PS: I tried adding if __name__ == '__main__': pool = Pool(12) pool.map(run, args) PS:我尝试添加if __name__ == '__main__': pool = Pool(12) pool.map(run, args)

Traceback (most recent call last): Traceback (most recent call last): "File """", line 1, in Traceback (most recent call last): Traceback (most recent call last):" "File """", line 1, in " "File ""c:\Python39\lib\rrultiprocessing\spawn.py"", line 116, in spawn_main exitcode = _main(fd, parent_sentinel)" "File ""c:\Python39\lib\rrultiprocessing\spawn.py"", line 125, in main" "File ""c:\Python39\lib\rnultiprocessing\spawn.py"", line 116, in spawn_main File """", line 1, in " Traceback (roost recent call last): "Traceback (roost recent call last): File """", line 1, in " "File ""c:\Python39\lib\rnultiprocessing\spawn.py'', line 125,in main prepare(preparation_data)" "File ""c:\Python39\lib\multiprocessing\spawn.py"",line 236,in prepare" _fixup_main_from_path(data['init_main_from_path']) "File ""c:\Python39\lib\multiprocessing\spawn.py"",line 287,in _fixup_main_from_path main_content = runpy.run_path(main_path," "File ""c:\Python39\lib\runpy.py'', line 268, in run_path return _run_module_c Traceback(最近一次调用最后一次):Traceback(最近一次调用最后一次):“文件”,第1行,在Traceback(最近一次调用最后一次):Traceback(最近一次调用最后一次):“文件”“”” , 第 1 行, 在 ""File ""c:\Python39\lib\rrultiprocessing\spawn.py"", 第 116 行, 在 spawn_main exitcode = _main(fd, parent_sentinel)" "File ""c:\Python39\lib\ rrultiprocessing\spawn.py"",第 125 行,在 main" "文件 ""c:\Python39\lib\rnultiprocessing\spawn.py"",第 116 行,在 spawn_main 文件中 """",第 1 行,在 "Traceback (roost 最近调用最后一次): "Traceback (roost 最近调用最后一次): 文件 """", line 1, in " "File ""c:\Python39\lib\rnultiprocessing\spawn.py'', line 125,in主要准备(preparation_data)“文件”“c:\Python39\lib\multiprocessing\spawn.py”,第236行,准备中“_fixup_main_from_path(数据['init_main_from_path'])”文件“”c:\Python39\lib \multiprocessing\spawn.py"", line 287,in _fixup_main_from_path main_content = runpy.run_path(main_path," "File ""c:\Python39\lib\runpy.py'', line 268, in run_path return _run_module_c ode(code, init_globals, run_name," "File ""c:\Python39\lib\runpy.py'', line 97, in _run_module_code exitcode = _main(fd, parent_sentinel)" "File ""c:\Python39\lib\multiprocessing\spawn.py"", line 116, in spawn_main File ""c:\Python39\lib\multiprocessing\spawn.py"", line 116, in spawn_main File ""'', line 1, in " "Traceback (most recent call last): File """", line 1, in " "File ""c:\Python39\lib\rrultiprocessing\spawn.py'', line 125, in main" ode(code, init_globals, run_name," "File ""c:\Python39\lib\runpy.py'',第 97 行,在 _run_module_code exitcode = _main(fd, parent_sentinel)" "File ""c:\Python39\lib \multiprocessing\spawn.py"",第 116 行,在 spawn_main 文件中 ""c:\Python39\lib\multiprocessing\spawn.py"",第 116 行,在 spawn_main 文件中 ""'',第 1 行,在 ""Traceback 中(最近一次调用最后一次):文件“”“”,第 1 行,在“文件”“c:\Python39\lib\rultiprocessing\spawn.py'',第 125 行,在 main”中

You must set the multiprocessing Contexts and start methods您必须设置多处理上下文和启动方法

For my case, I had to utilize the context 'fork'就我而言,我不得不使用上下文“fork”

ctx = multiprocessing.get_context('fork')
work_queue    = ctx.Queue()
results_queue = ctx.Queue()
...
 workers = get_worker_processes(
    _process_data,
    (task_function, work_queue, results_queue),
    nproc=nproc,
)          
.....
workers = [
    ctx.Process(target=f, args=args) for _ in range(num_procs)
]   

Follow the guidance given in the Python multiprocessing documentation for the link referenced.请遵循 Python 多处理文档中给出的指南以获取所引用的链接。 Notice the change in defaults.注意默认值的变化。

Contexts and start methods Depending on the platform, multiprocessing supports three ways to start a process.上下文和启动方法 根据平台,多处理支持三种启动进程的方法。 These start methods are这些启动方法是

spawn The parent process starts a fresh python interpreter process. spawn 父进程启动一个新的 python 解释器进程。 The child process will only inherit those resources necessary to run the process object's run() method.子进程将仅继承运行进程对象的 run() 方法所需的那些资源。 In particular, unnecessary file descriptors and handles from the parent process will not be inherited.特别是,不会继承父进程中不必要的文件描述符和句柄。 Starting a process using this method is rather slow compared to using fork or forkserver.与使用 fork 或 forkserver 相比,使用这种方法启动进程相当慢。

Available on Unix and Windows.适用于 Unix 和 Windows。 The default on Windows and macOS. Windows 和 macOS 上的默认设置。

fork The parent process uses os.fork() to fork the Python interpreter. fork 父进程使用 os.fork() 来 fork Python 解释器。 The child process, when it begins, is effectively identical to the parent process.子进程在开始时实际上与父进程相同。 All resources of the parent are inherited by the child process.父进程的所有资源都由子进程继承。 Note that safely forking a multithreaded process is problematic.请注意,安全地分叉多线程进程是有问题的。

Available on Unix only.仅在 Unix 上可用。 The default on Unix. Unix 上的默认值。

forkserver When the program starts and selects the forkserver start method, a server process is started. forkserver 当程序启动并选择forkserver启动方式时,一个服务器进程被启动。 From then on, whenever a new process is needed, the parent process connects to the server and requests that it fork a new process.从那时起,每当需要一个新进程时,父进程都会连接到服务器并请求它派生一个新进程。 The fork server process is single threaded so it is safe for it to use os.fork(). fork 服务器进程是单线程的,因此使用 os.fork() 是安全的。 No unnecessary resources are inherited.没有不必要的资源被继承。

Available on Unix platforms which support passing file descriptors over Unix pipes.在支持通过 Unix 管道传递文件描述符的 Unix 平台上可用。

Changed in version 3.8: On macOS, the spawn start method is now the default.在 3.8 版更改: 在 macOS 上,spawn start 方法现在是默认值。 The fork start method should be considered unsafe as it can lead to crashes of the subprocess. fork start 方法应该被认为是不安全的,因为它可能导致子进程崩溃。 See bpo-33725.请参阅 bpo-33725。

Changed in version 3.4: spawn added on all unix platforms, and forkserver added for some unix platforms.在 3.4 版中更改:在所有 unix 平台上添加了 spawn,并为一些 unix 平台添加了 forkserver。 Child processes no longer inherit all of the parents inheritable handles on Windows.子进程不再继承 Windows 上的所有父进程可继承句柄。

声明:本站的技术帖子网页,遵循CC BY-SA 4.0协议,如果您需要转载,请注明本站网址或者原文地址。任何问题请咨询:yoyou2525@163.com.

 
粤ICP备18138465号  © 2020-2024 STACKOOM.COM