How can I use this function in multiprocessing?

Question

I want to apply this function in multiprocessing:

import multiprocessing
import numpy as np

a = np.random.randint(1,4, size=(10,10,10))

def function(input_field,[num1,num2,num3]):
    pass

if __name__ == '__main__':
    pool = multiprocessing.Pool(processes=4)
    test_list = [(a,[27.5, 27.5, 25]), (b,[27.5, 27.5, 25]), (c,[27.5, 27.5, 25]), (d,[27.5, 27.5, 25])]
    results = pool.map(function, test_list)
    pool.close()
    pool.join()    

How can I apply function(input_field,[num1,num2,num3]) to multiprocessing?

Answer 1

You have 2 options based on the structure of your test_list :

1. modify your function to support a single argument and unpack it inside the function and keep using map :

    def function(arg): input_field, l = arg

2. use starmap and fix your function definition, because the structure of test_list is what starmap is expecting to get:

    def function(input_field, l): ... results = pool.starmap(function, test_list)