繁体   English   中英

Python 星图多处理和 __main__ 问题

[英]Python multiprocessing with starmap and issue with __main__

我正在尝试使用多个参数进行小型多处理

任务类型 1:

import multiprocessing  as mp
import pandas as pd
import os,sys
print("Libs Loaded..!!! ")
listoflists = [[1,2,3,4],[4,5,6,7],[7,8,9,10],[10,11,13,14]]
listoftuples = [tuple(i) for i in listoflists]
print("Length of Tuples : ",len(listoftuples))

def map_function(combo):
    a = combo[0]
    b = combo[1]
    c = combo[2]
    print((a + b + c))
    return (a + b + c)

def doit():
    try:
        print("IN main")
        p = mp.Pool(processes=2)
        # results= p.map(map_function, listoftuples)
        results = p.starmap(map_function,listoftuples)
        print(results)
        print("Done!!")
    except Exception as e:
        exc_type, exc_obj, exc_tb = sys.exc_info()
        fname = os.path.split(exc_tb.tb_frame.f_code.co_filename)[1]
        print(exc_type, fname, exc_tb.tb_lineno)

# if '__name__' == '__main__': 
doit()     # running without entry point

TaskType-1 错误:

Libs Loaded..!!!
Length of Tuples :  4
IN main
Libs Loaded..!!!
Length of Tuples :  4
IN main
<class 'RuntimeError'> testformp.py 20
<class 'TypeError'> testformp.py 22
Libs Loaded..!!!
Length of Tuples :  4
IN main
<class 'RuntimeError'> testformp.py 20

不确定这个运行时错误以及为什么它多次进入 doit() function。 Multipeprocessing 在此 function 中定义,但在这里它一次又一次地调用父级 function ..不确定我在这里缺少什么理解?

任务类型 2:

if '__name__' == '__main__':
    doit()  # running from entry point

output 对于 TaskType-2:

Libs Loaded..!!!
Length of Tuples :  4
    

它没有显示任何错误,也没有在内部执行任何任务。 为什么会这样?

做出我建议的两项更改,我最终得到了这个:

import multiprocessing  as mp
import os,sys
#print("Libs Loaded..!!! ")
listoflists = [[1,2,3,4],[4,5,6,7],[7,8,9,10],[10,11,13,14]]
listoftuples = [tuple(i) for i in listoflists]
#print("Length of Tuples : ",len(listoftuples))

def map_function(*combo):
    a = combo[0]
    b = combo[1]
    c = combo[2]
    #print((a + b + c))
    return (a + b + c)

def doit():
    try:
        print("IN main")
        p = mp.Pool(processes=2)
        #results= p.map(map_function, listoftuples)
        results = p.starmap(map_function,listoftuples)
        print(results)
        print("Done!!")
    except Exception as e:
        exc_type, exc_obj, exc_tb = sys.exc_info()
        fname = os.path.split(exc_tb.tb_frame.f_code.co_filename)[1]
        print(exc_type, fname, exc_tb.tb_lineno)

if __name__ == '__main__':
    doit()     # running without entry point

output 是这样的:

IN main
[6, 15, 24, 34]
Done!!

暂无
暂无

声明:本站的技术帖子网页,遵循CC BY-SA 4.0协议,如果您需要转载,请注明本站网址或者原文地址。任何问题请咨询:yoyou2525@163.com.

 
粤ICP备18138465号  © 2020-2024 STACKOOM.COM