Why do I get error trying to cast np.array(some_list) ValueError: could not broadcast input array

Question

I'm trying to create np.array of np.arrays of different shape. I need it because in the next part I will sum up this large np.array with some delta_array that has the same shape. Like matrices sum self.weights += delta_weights And I will be working with each array inside it separately as well. Or for example I will need to multiply element wise all arrays by some number.

Can't figure out what's wrong :( Please, help me. Code creates list of random np.arrays of different shape. So here is my code:

weights = []
for i in range(1, len(self.layers)):
    weights.append(np.random.rand(self.layers[i-1] + 1, self.layers[i]))
print(type(weights))
print([(type(w), w.shape) for w in weights])
#error here with layers = [2,2,1] or [3,3,1] etc
self.weights = np.array(weights)

Output: For self.layers=[2, 2, 1]

<class 'list'>
[(<class 'numpy.ndarray'>, (3, 2)), (<class 'numpy.ndarray'>, (3, 1))]
Traceback (most recent call last):
line 20, in <module>
 run()
line 8, in run
 net.init_weights()
line 71, in init_weights
 self.weights = np.array(weights)
ValueError: could not broadcast input array from shape (3,2) into shape (3)

For [2, 3, 1] everything is okay:

<class 'list'>
[(<class 'numpy.ndarray'>, (3, 3)), (<class 'numpy.ndarray'>, (4, 1))]

For [3, 3, 1] same story as for [2, 2, 1] - error

For [3, 7, 1] or [3, 2, 1] everything is okay.

//It's matrices of weights for gradient descent for machine learning.

Answer 1

Ok, you're trying to create an array of matrices, so that you can use array operations (like += ) later on.

You can declare an ndarray to hold another ndarray , and then fill it with the matrices (which are simply two dimensional ndarrays):

weights = np.empty(len(self.layers), dtype=np.ndarray)
for i in range(1, len(self.layers)):
    weights[i] = np.random.rand(self.layers[i-1] + 1, self.layers[i])

Keep in mind that this is definitely not a three dimensional array: it's purely an array of two dimensional arrays.

The reason your code works for [2, 3, 1] is that the array with one dimension of 1 gets broadcasted; that's purely accidental, and not what you want, as far as I understand.

Answer 2

When you give np.array a list of arrays, it tries to combine them into one array (with a higher ndim ). Only if it can't combine them will it create an array of dtype object with one array in each slot.

In the [2,3,1] case the 2 arrays are (3,3) and (4,1) . Since there's no way to 'stack' these, it creates a 2 element array containing these 2.

In the [3,3,1] case the 2 arrays are (4,3) and (4,1) . Due to an overlap in dimensions (4 rows) it appears to be trying to join them into one array, possibly (2,4,?) , but isn't quite succeeding. In your version it raises an error, in my version it creates a (2,4) array of Nones.

Evert's solution is a general purpose one that correctly handles these confusing case. It works even if the 2 arrays in weights were the same size.

Say, for example if your weights contained 2 (4,3) arrays. np.arrays(weights) would produce a (2,4,3) array. Evert's solution would still produce a (2,) array containing 2 (4,3) arrays.

Looks like there's a bug (or two) in different numpy versions. Python3 with 1.8.0 numpy handled the [2,2,1] case just fine.

A simpler test case would be:

np.array([np.ones((2,2)), np.zeros((2,1))])

Playing around the dimensions of the 2 arrays. It should produce a 2 element array in most cases, and a (2,n,m) array if the 2 dimensions are the same.