Python2: multiprocessing using a Pool of workers with multiple arguments

Question

I want to try parallel computing in python-2.x using multiprocessing.Pool .

I came up with the following simple code. Unfortunately, I was not able to produce any error message.

Can someone point me into the right direction of what might be wrong with my code?

import numpy as np
import multiprocessing as mp
import timeit

def fun(i,j):
    return i+j

num=2

num_cores = mp.cpu_count()
p = mp.Pool(num_cores)

results = np.array([])
tasks = np.array([(i, j) for i in range(0,num) for j in range(0, num)])

if __name__ == '__main__':
    results = np.array(p.map(fun,tasks))
    print results

Answer 1

The main problem you have here is that the tuples you want to pass as arguments to the workers are wrapped in a numpy.ndarray and thus it is just one argument as the numpy.ndarray does not get unfolded.

---

This should work for you:

from __future__ import print_function
import numpy as np
import multiprocessing as mp
import timeit

def fun(ij):
    s = sum(ij)
    print('{0} + {1} = {2}'.format(ij[0], ij[1], s))
    return s

num=2

num_cores = mp.cpu_count()
p = mp.Pool(num_cores)

results = np.array([])
tasks = np.array([(i, j) for i in range(0,num) for j in range(0, num)])

if __name__ == '__main__':
    results = np.array(p.map(fun,tasks))
    print(results)

The output of this script is:

0 + 0 = 0
0 + 1 = 1
1 + 0 = 1
1 + 1 = 2
[0 1 1 2]

---

I added the from __future__ import print_function statement, see here what it is good for. And here is the according PEP 3105 -- Make print a function .

---

Final remarks: A more throughout solution can be found here from user senderle.