简体   繁体   English

Python 2至Python 3:无法进行多重处理

[英]Python 2 to Python 3: can't get multiprocessing to work

I'm moving my code from python 2.7 to python 3.5 and cannot get my multiprocessing code to work, something like the following code. 我将代码从python 2.7移至python 3.5,无法使我的多处理代码正常工作,类似于以下代码。 "somemodule.py" in the same directory as the main script . 与主脚本位于同一目录中的“ somemodule.py”

import multiprocessing as mp

# somemodule is in the same folder as this script
import somemodule

def foo(bar):
    print(bar)
    return

if __name__ == '__main__':
    bar = ("alice","bob")
    pool = mp.Pool(processes=2)
    pool.map(foo, bar)
    pool.close()

The traceback is 追溯是

Traceback (most recent call last):
  File "<string>", line 1, in <module>
  File "C:\Anaconda3\lib\multiprocessing\spawn.py", line 106, in spawn_main
    exitcode = _main(fd)
  File "C:\Anaconda3\lib\multiprocessing\spawn.py", line 115, in _main
    prepare(preparation_data)
  File "C:\Anaconda3\lib\multiprocessing\spawn.py", line 226, in prepare
    _fixup_main_from_path(data['init_main_from_path'])
  File "C:\Anaconda3\lib\multiprocessing\spawn.py", line 278, in _fixup_main_from_path
    run_name="__mp_main__")
  File "C:\Anaconda3\lib\runpy.py", line 240, in run_path
    pkg_name=pkg_name, script_name=fname)
  File "C:\Anaconda3\lib\runpy.py", line 96, in _run_module_code
    mod_name, mod_spec, pkg_name, script_name)
  File "C:\Anaconda3\lib\runpy.py", line 85, in _run_code
    exec(code, run_globals)
  File "C:\Users\hobboy\Documents\project\pool_test3.py", line 2, in <module>
    import somemodule
ImportError: No module named 'somemodule'

The code works in the Python Console of Spyder, but not IPython (working with Anaconda3 2.5.0). 该代码可在Spyder的Python控制台中使用,但不能在IPython(与Anaconda3 2.5.0一起使用)中使用。 The worker processes are unable to import somemodule but the main process can . 工作进程无法导入somemodule但主进程可以 Is it something to do with import somemodule being inadequate, or something else? import somemodule不足或其他原因有关吗? I've read into the Python 3 import syntax a bit, but I'm still not sure. 我已经读了一些Python 3导入语法,但是我仍然不确定。

I've made a post on the Anaconda community forums but that's more in regards to if it's an issue with Anaconda rather than the code itself. 我已经在Anaconda社区论坛上发表了一篇文章,但这更多是关于Anaconda的问题,而不是代码本身。


Additional information: 附加信息:

For main process (non worker) for Python 2.7/3.5, sys.path[0]='' regardless of console/python version 对于Python 2.7 / 3.5的主进程(非工作程序),无论控制台/ python版本如何, sys.path[0]=''

What's interesting is that sys.path[0] varies for the worker processes: 有趣的是sys.path [0]因工作进程而异:

Spyder 2.3.8 (Python 2.7) Spyder 2.3.8(Python 2.7)

  • Python Console: '' (works) Python控制台:”(有效)
  • IPython Console: '' (works) IPython控制台:”(有效)

Spyder 2.3.8 (Python 3.5) Spyder 2.3.8(Python 3.5)

  • Python Console: 'C:\\Users\\hobboy\\Documents\\project' (works) Python控制台:“ C:\\ Users \\ hobboy \\ Documents \\ project”(有效)
  • IPython Console: 'C:\\WINDOWS\\system32' (doesn't work) IPython控制台:“ C:\\ WINDOWS \\ system32”(无效)

However the main (non worker) process for both Python 2 and 3 being sys.path='' . 但是,Python 2和3的主要(非工作程序)进程均为sys.path='' Why is sys.path[0] being set differently (and inconsistently) in Python 3 为什么在Python 3中将sys.path [0]设置为不同(且不一致)

Look at what you get when you add the following code on the very first line of your script: 查看在脚本的第一行添加以下代码时得到的结果:

import sys; print(sys.path)

The output should include the directory where your "somemodule" is. 输出应包括“ somemodule”所在的目录。 If it's not listed, you can explicitly manipulate your environment by appending the directory to the sys.path list (careful escaping backslashes) before importing the module. 如果未列出,则可以在导入模块之前,通过将目录附加到sys.path列表(小心地转义反斜杠)来显式地操作环境。

Another alternative is making sure the python interpreter is run directly from that folder, so that the "working directory" includes somemodule.py. 另一种选择是确保直接从该文件夹运行python解释器,以便“工作目录”包含somemodule.py。

声明:本站的技术帖子网页,遵循CC BY-SA 4.0协议,如果您需要转载,请注明本站网址或者原文地址。任何问题请咨询:yoyou2525@163.com.

 
粤ICP备18138465号  © 2020-2024 STACKOOM.COM