Numpy parallelization for more than one node?

Question

It is possible to use more cores for certain numpy operations like np.dot . Is it also possible to use more than one node?

Answer 1

Numpy is not designed for easy splitting into multiple nodes. You have to perform the split manually, ie split into subarrays processed by different nodes (if possible at all for your operation).

You may be using multiple cores, depending on the underlying library [1] .

Alternatively, you can look at Blaze and Dask modules for Numpy type of operations with multicore support.

Answer 2

For splitting work on several process you have to implement Process (not Thread)!

Example (extract from official doc):

from multiprocessing import Process

def f(name):
    print('hello', name)

if __name__ == '__main__':
    p = Process(target=f, args=('bob',))
    p.start()
    p.join()

See for detail the official documentation

Additionally, this topic may be of interest regarding the numpy optimized syntax related to your situation.

EDIT As stated on this link , you can use this to split over cores or nodes!