multiprocessing.Array (python): float expected instead of numpy.ndarray instance

Question

I have followed online manual on multiprocessing already. I think I have given it an array to mp.Array. Why does it expect a float instead of an array?

My script -

import multiprocessing as mp
import numpy as np

pdb_num = 1000

fitting_theta = mp.Array('d', np.zeros((pdb_num,pdb_num)))
fitting_deviation = mp.Array('d', np.zeros((pdb_num,pdb_num)))

And it gives an error -

float expected instead of numpy.ndarray instance

Thanks.

Edit :

Complete traceback -

  File "/lustre/beagle2/danielhskerr/python/min_sq_fitnewres.py", line 70, in <module>
fitting_theta = mp.Array('d', np.zeros((pdb_num,pdb_num)))
  File "/soft/python/2.7/2.7.1/lib/python2.7/multiprocessing/__init__.py", line 256, in Array
return Array(typecode_or_type, size_or_initializer, **kwds)
  File "/soft/python/2.7/2.7.1/lib/python2.7/multiprocessing/sharedctypes.py", line 87, in Array
  obj = RawArray(typecode_or_type, size_or_initializer)
  File "/soft/python/2.7/2.7.1/lib/python2.7/multiprocessing/sharedctypes.py", line 61, in RawArray
result.__init__(*size_or_initializer)
TypeError:  float expected instead of numpy.ndarray instance

Answer 1

Please read the documentation to multiprocessing carefully: https://docs.python.org/2/library/multiprocessing.html#multiprocessing.Array

The types available for marshalling between processes are quite limited, and thus you can't use an arbitrary object/type (such as an numpy-array) for them.

This might be an answer to your problem

Use numpy array in shared memory for multiprocessing

however the requirement to lock access might influence the performance unduly. Use a normal array with floats instead for spawning your processes, and then build numpy-arrays from them. But if one of these solutions is a fit to your problem depends on that very problem.