[英]How to run a function parallelly in python?
I have the function Test below我有下面的 function 测试
import concurrent.futures
from multiprocessing import Pool
def Test(n):
print(n)
which i want to run parallelly so I do this我想并行运行所以我这样做
li = ['a','b', 'c']
with concurrent.futures.ProcessPoolExecutor() as executor:
executor.map(Test, li )
but I do not see the output ie printing a, b, c但我没有看到 output 即打印 a、b、c
I am running it on Jupyter notebook.我在 Jupyter notebook 上运行它。 not sure if that is the issue.
不确定这是否是问题所在。 I do not see any output. Although if i call test function individually it runs fine.
我没有看到任何 output。虽然如果我单独调用测试 function 它运行良好。
I am running it on Windows OS我在 Windows 操作系统上运行它
I would normally issue a close vote on this as a duplicate of this question , but now that we have finally established that you are running under Windows, you have two issues with your code.我通常会对此问题进行一次投票,作为这个问题的副本,但现在我们终于确定您在 Windows 下运行,您的代码有两个问题。
Test
from an external file to use multiprocessing with Jupyter Notebook successfully.Test
,才能成功使用 Jupyter Notebook 进行多处理。 But multiprocessing code is best not run from Jupyter Notebook because of this complication and also because terminal output from your subprocesses that would normally be captured and written below the cell is instead written to the Jupyter Notebook logging window (see below).if __name__ == '__main__':
block.if __name__ == '__main__':
块中。 In fact, you should place any code at global scope that is not required by your worker function in such a block for improved efficiency as it would otherwise be needlessly executed by every process in your pool as part of its initialization. File test.py in same directory as your.ipynb file文件test.py与 your.ipynb 文件在同一目录中
def Test(n):
print(n)
And your cell还有你的手机
if __name__ == '__main__':
from test import Test
import concurrent.futures
li = ['a','b', 'c']
with concurrent.futures.ProcessPoolExecutor() as executor:
executor.map(Test, li )
Here is the output from the Jupyter Notebook logging window (here I am actually usually using Jupyter Lab, which is built on top of Jupyter Notebook):这是来自 Jupyter Notebook 的 output 日志记录 window(这里我实际上通常使用 Jupyter Lab,它建立在 Jupyter Notebook 之上):
[I 06:08:25.599 LabApp] JupyterLab extension loaded from c:\program files\python38\lib\site-packages\jupyterlab
[I 06:08:25.600 LabApp] JupyterLab application directory is c:\program files\python38\share\jupyter\lab
[I 06:08:25.702 LabApp] Serving notebooks from local directory: C:\Booboo\test
[I 06:08:25.703 LabApp] Jupyter Notebook 6.1.5 is running at:
[I 06:08:25.706 LabApp] http://localhost:8888/?token=156f2a52135726b662215b349b4047dfea557c17d1acb366
[I 06:08:25.706 LabApp] or http://127.0.0.1:8888/?token=156f2a52135726b662215b349b4047dfea557c17d1acb366
[I 06:08:25.707 LabApp] Use Control-C to stop this server and shut down all kernels (twice to skip confirmation).
[C 06:08:26.082 LabApp]
To access the notebook, open this file in a browser:
file:///C:/Users/Booboo/AppData/Roaming/jupyter/runtime/nbserver-13472-open.html
Or copy and paste one of these URLs:
http://localhost:8888/?token=156f2a52135726b662215b349b4047dfea557c17d1acb366
or http://127.0.0.1:8888/?token=156f2a52135726b662215b349b4047dfea557c17d1acb366
[I 06:08:39.827 LabApp] Build is up to date
[I 06:08:58.745 LabApp] Kernel started: bac1f587-2c31-45e5-b1ca-7f56dd1929ba, name: python3
a
c
b
If Test
were instead:如果
Test
是:
def Test(n):
return n * n
And your cell:还有你的手机:
if __name__ == '__main__':
from test import Test
import concurrent.futures
li = [1, 2, 3]
with concurrent.futures.ProcessPoolExecutor() as executor:
print(list(executor.map(Test, li)))
Then the printing would be done by your main process and you would see the output beneath the cell.然后打印将由您的主进程完成,您将在单元格下方看到 output。
You should also look at PEP 8 – Style Guide for Python Code when you have a chance.如果有机会,您还应该查看PEP 8 – Python 代码的样式指南。 Function names are usually not started with uppercase letters.
Function 名称通常不以大写字母开头。
声明:本站的技术帖子网页,遵循CC BY-SA 4.0协议,如果您需要转载,请注明本站网址或者原文地址。任何问题请咨询:yoyou2525@163.com.