[英]Multiprocessing in Python Issue
I'm trying multiprocessing in python but can't seem to get it to work.我正在 python 中尝试多处理,但似乎无法让它工作。
The input file is as follows:输入文件如下:
And the code is as follows:代码如下:
import pandas as pd
import multiprocessing
import time
import datetime
start_time = datetime.datetime.now()
df_main = []
df_main = pd.read_csv("data.csv")
df_file = []
def growth_calculator(Type):
values = [Type]
global df_temp, df_file
df_temp = df_main[df_main.Type.isin(values)]
df_temp = df_temp[['Company', 'Type']]
print(df_temp)
time.sleep(10)
if __name__ == '__main__':
multiprocessing.Process(target=growth_calculator('Quarterly'))
multiprocessing.Process(target=growth_calculator('Annual'))
multiprocessing.Process(target=growth_calculator('Monthly'))
end_time = datetime.datetime.now()
print("Time Taken -", end_time-start_time)
The output should take around 10-11 seconds, but it's taking 30 seconds. output 应该需要大约 10-11 秒,但需要 30 秒。 So, clearly, multiprocessing isn't working.因此,显然,多处理不起作用。
Could you please point me to the right direction?你能指出我正确的方向吗?
Thanks in advance!提前致谢!
you need to pass target arguments as args=
keyword for the Process init (see https://docs.python.org/3/library/multiprocessing.html#multiprocessing.Process ).您需要将目标 arguments 作为args=
关键字传递给 Process init(参见https://docs.python.org/3/library/multiprocessing.html#.multiprocessing )。 Otherwise your function is evaluated before instantiating process, which leads to single-process performance.否则,在实例化进程之前评估您的 function,这会导致单进程性能。
Something like this:像这样的东西:
import pandas as pd
import multiprocessing
import time
import datetime
start_time = datetime.datetime.now()
def growth_calculator(Type):
print(Type)
time.sleep(10)
if __name__ == '__main__':
p1 = multiprocessing.Process(target=growth_calculator,args=('Quarterly',))
p2 = multiprocessing.Process(target=growth_calculator,args=('Annual',))
p3 = multiprocessing.Process(target=growth_calculator,args=('Monthly',))
p1.start()
p2.start()
p3.start()
print('started')
p1.join()
p2.join()
p3.join()
end_time = datetime.datetime.now()
print("Time Taken -", end_time-start_time)
声明:本站的技术帖子网页,遵循CC BY-SA 4.0协议,如果您需要转载,请注明本站网址或者原文地址。任何问题请咨询:yoyou2525@163.com.