I was trying to make small multiprocessing with multiple parameters
TaskType-1:
import multiprocessing as mp
import pandas as pd
import os,sys
print("Libs Loaded..!!! ")
listoflists = [[1,2,3,4],[4,5,6,7],[7,8,9,10],[10,11,13,14]]
listoftuples = [tuple(i) for i in listoflists]
print("Length of Tuples : ",len(listoftuples))
def map_function(combo):
a = combo[0]
b = combo[1]
c = combo[2]
print((a + b + c))
return (a + b + c)
def doit():
try:
print("IN main")
p = mp.Pool(processes=2)
# results= p.map(map_function, listoftuples)
results = p.starmap(map_function,listoftuples)
print(results)
print("Done!!")
except Exception as e:
exc_type, exc_obj, exc_tb = sys.exc_info()
fname = os.path.split(exc_tb.tb_frame.f_code.co_filename)[1]
print(exc_type, fname, exc_tb.tb_lineno)
# if '__name__' == '__main__':
doit() # running without entry point
Error for TaskType-1:
Libs Loaded..!!!
Length of Tuples : 4
IN main
Libs Loaded..!!!
Length of Tuples : 4
IN main
<class 'RuntimeError'> testformp.py 20
<class 'TypeError'> testformp.py 22
Libs Loaded..!!!
Length of Tuples : 4
IN main
<class 'RuntimeError'> testformp.py 20
Not sure this runtime Error and why it goes in into doit() function multiple times. Multipeprocessing is defined within this function but here it is calling the parent function again and again..Not sure what I am missing here to understand?
TaskType-2:
if '__name__' == '__main__':
doit() # running from entry point
output for TaskType-2:
Libs Loaded..!!!
Length of Tuples : 4
It shows no errors nor it performs any task inside. Why this is so?
Making the two changes I suggested I end up with this:
import multiprocessing as mp
import os,sys
#print("Libs Loaded..!!! ")
listoflists = [[1,2,3,4],[4,5,6,7],[7,8,9,10],[10,11,13,14]]
listoftuples = [tuple(i) for i in listoflists]
#print("Length of Tuples : ",len(listoftuples))
def map_function(*combo):
a = combo[0]
b = combo[1]
c = combo[2]
#print((a + b + c))
return (a + b + c)
def doit():
try:
print("IN main")
p = mp.Pool(processes=2)
#results= p.map(map_function, listoftuples)
results = p.starmap(map_function,listoftuples)
print(results)
print("Done!!")
except Exception as e:
exc_type, exc_obj, exc_tb = sys.exc_info()
fname = os.path.split(exc_tb.tb_frame.f_code.co_filename)[1]
print(exc_type, fname, exc_tb.tb_lineno)
if __name__ == '__main__':
doit() # running without entry point
And the output is this:
IN main
[6, 15, 24, 34]
Done!!
The technical post webpages of this site follow the CC BY-SA 4.0 protocol. If you need to reprint, please indicate the site URL or the original address.Any question please contact:yoyou2525@163.com.