[英]Why does my python multiprocessing script run on Windows but not on Linux?
I've successfully implemented a multiprocessed script on Windows, but the same script launches a "RuntimeError: already started" on linux and stops the execution. 我已经在Windows上成功实现了多进程脚本,但是相同的脚本在Linux上启动了“ RuntimeError:已经启动”并停止了执行。 The script consists of the following "main.py" (omitted some part for readability):
该脚本由以下“ main.py”组成(为便于阅读,省略了一部分):
from multiprocessing import freeze_support
if __name__ == '__main__':
#MULTIPROCESSING STUFF
freeze_support()
#DO SOME STUFF
#Call my multiprocessing-function in other module
mod2.func(tileTS[0], label, areaconst)
And the "mod2.py" module: 和“ mod2.py”模块:
import numpy as np
from multiprocessing import Pool
from functools import partial
import os, time
def func(ts, label, areaconst):
#SETTING UP/LOADING SOME VARIABLES
for idx in totImgs:
img_ = myList[idx]
p = Pool(2)
result = p.map( partial(_mp_avg, var1=var1_, img=img_), range(totObjs) )
p.close()
p.join()
#MANAGE RESULTING VARIABLES
return None
def _mp_avg(idx, img, var1):
num = idx + 1
arr = img[var1==num]
if np.isnan(arr).any():
return np.nan
else:
return np.sum( arr )
This error is launched when the script executes the "Pool.map" function/class (dunno tbh). 当脚本执行“ Pool.map”函数/类(dunno tbh)时,将启动此错误。 The same code works flawlessly on Windows.
相同的代码在Windows上可以完美运行。
I'm using Ubuntu 18.04 and launching the python 3.6.7 script from Visual Studio Code. 我正在使用Ubuntu 18.04,并从Visual Studio Code启动python 3.6.7脚本。
As pointed out by @Darkonaut, Visual Studio Code uses ptvsd as debugger, which isn't fork-save ( https://github.com/Microsoft/ptvsd/issues/1046#issuecomment-443339930 ). 正如@Darkonaut指出的那样,Visual Studio Code使用ptvsd作为调试器,而不是进行分叉保存( https://github.com/Microsoft/ptvsd/issues/1046#issuecomment-443339930 )。 Since on linux the default process spawn method is "os.fork()", the script will generate a RuntimeError if executed from within VSCode.
由于在Linux上,默认的进程生成方法是“ os.fork()”,因此,如果从VSCode中执行该脚本,则会生成RuntimeError。 This will not happen on Windows.
在Windows上不会发生这种情况。 Solutions on Linux are:
Linux上的解决方案是:
Change start-method by inserting once the following line after the main function call: 通过在主函数调用后插入以下行一次来更改启动方法:
multiprocessing.set_start_method("spawn")
Edit code with VSCode and launch from Terminal. 使用VSCode编辑代码并从Terminal启动。
Change IDE. 更改IDE。
Wait for fork-save debugger update, which is supposedly under work. 等待fork-save调试器更新,该更新可能正在工作中。
Check the following link for further information about the problem: https://docs.python.org/3/library/multiprocessing.html#contexts-and-start-methods 检查以下链接以获取有关该问题的更多信息: https : //docs.python.org/3/library/multiprocessing.html#contexts-and-start-methods
声明:本站的技术帖子网页,遵循CC BY-SA 4.0协议,如果您需要转载,请注明本站网址或者原文地址。任何问题请咨询:yoyou2525@163.com.