简体   繁体   English

Python 星图多处理和 __main__ 问题

[英]Python multiprocessing with starmap and issue with __main__

I was trying to make small multiprocessing with multiple parameters我正在尝试使用多个参数进行小型多处理

TaskType-1:任务类型 1:

import multiprocessing  as mp
import pandas as pd
import os,sys
print("Libs Loaded..!!! ")
listoflists = [[1,2,3,4],[4,5,6,7],[7,8,9,10],[10,11,13,14]]
listoftuples = [tuple(i) for i in listoflists]
print("Length of Tuples : ",len(listoftuples))

def map_function(combo):
    a = combo[0]
    b = combo[1]
    c = combo[2]
    print((a + b + c))
    return (a + b + c)

def doit():
    try:
        print("IN main")
        p = mp.Pool(processes=2)
        # results= p.map(map_function, listoftuples)
        results = p.starmap(map_function,listoftuples)
        print(results)
        print("Done!!")
    except Exception as e:
        exc_type, exc_obj, exc_tb = sys.exc_info()
        fname = os.path.split(exc_tb.tb_frame.f_code.co_filename)[1]
        print(exc_type, fname, exc_tb.tb_lineno)

# if '__name__' == '__main__': 
doit()     # running without entry point

Error for TaskType-1: TaskType-1 错误:

Libs Loaded..!!!
Length of Tuples :  4
IN main
Libs Loaded..!!!
Length of Tuples :  4
IN main
<class 'RuntimeError'> testformp.py 20
<class 'TypeError'> testformp.py 22
Libs Loaded..!!!
Length of Tuples :  4
IN main
<class 'RuntimeError'> testformp.py 20

Not sure this runtime Error and why it goes in into doit() function multiple times.不确定这个运行时错误以及为什么它多次进入 doit() function。 Multipeprocessing is defined within this function but here it is calling the parent function again and again..Not sure what I am missing here to understand? Multipeprocessing 在此 function 中定义,但在这里它一次又一次地调用父级 function ..不确定我在这里缺少什么理解?

TaskType-2:任务类型 2:

if '__name__' == '__main__':
    doit()  # running from entry point

output for TaskType-2: output 对于 TaskType-2:

Libs Loaded..!!!
Length of Tuples :  4
    

It shows no errors nor it performs any task inside.它没有显示任何错误,也没有在内部执行任何任务。 Why this is so?为什么会这样?

Making the two changes I suggested I end up with this:做出我建议的两项更改,我最终得到了这个:

import multiprocessing  as mp
import os,sys
#print("Libs Loaded..!!! ")
listoflists = [[1,2,3,4],[4,5,6,7],[7,8,9,10],[10,11,13,14]]
listoftuples = [tuple(i) for i in listoflists]
#print("Length of Tuples : ",len(listoftuples))

def map_function(*combo):
    a = combo[0]
    b = combo[1]
    c = combo[2]
    #print((a + b + c))
    return (a + b + c)

def doit():
    try:
        print("IN main")
        p = mp.Pool(processes=2)
        #results= p.map(map_function, listoftuples)
        results = p.starmap(map_function,listoftuples)
        print(results)
        print("Done!!")
    except Exception as e:
        exc_type, exc_obj, exc_tb = sys.exc_info()
        fname = os.path.split(exc_tb.tb_frame.f_code.co_filename)[1]
        print(exc_type, fname, exc_tb.tb_lineno)

if __name__ == '__main__':
    doit()     # running without entry point

And the output is this: output 是这样的:

IN main
[6, 15, 24, 34]
Done!!

声明:本站的技术帖子网页,遵循CC BY-SA 4.0协议,如果您需要转载,请注明本站网址或者原文地址。任何问题请咨询:yoyou2525@163.com.

 
粤ICP备18138465号  © 2020-2024 STACKOOM.COM