为什么使用 numpy.reshape 的数组未正确存储？

Question

Let's consider an n-by-m array of values that I want to reshape into n*m-by-1 array for storage and plotting. The n-by-m array is the result of some analyses that takes place in a for loop that gets written over each iteration.

Each time I construct the n*m-by-1 array, I want to store the stacked columns into a separate variable and then increment the column number so I eventually end up with an array that is n*m-by-k where k is how many separate Excel files I'm working with.

The code below will replicate the error and illustrate the point.

import numpy as np

number = [2,4]

new_a = np.zeros((3,6))
vec_for_plot = np.zeros((3,6))

a = np.random.random((3,6))

i = 0

for n in range(0,len(number)):
    new_a = a + number[n]
    vec_for_plot = np.reshape(new_a,(-1,1))
    i = i + 1

If I try to assemble the n*m-by-1 array into the vec_for_plot variable using:

vec_for_plot[:,i] = np.reshape(new_a,(-1,1))

then the following error shows up.

ValueError: could not broadcast input array from shape (18,1) into shape (18)

I've tried np.column_stack, switching the indices, initializing empty matrices, matrices of zeros, etc.

Can someone please provide some guidance on where my indexing is going wrong?

Answer 1

In [40]: arr = np.zeros((3,6))

When I index a 'column' of this array, I get a 1d array, shape (3,):

In [41]: arr[:,0]                 
Out[41]: array([0., 0., 0.])
In [42]: _.shape
Out[42]: (3,)     

I can assign another (3,) shape array to it:

In [43]: arr[:,0] = np.arange(3)
In [44]: arr
Out[44]: 
array([[0., 0., 0., 0., 0., 0.],
       [1., 0., 0., 0., 0., 0.],
       [2., 0., 0., 0., 0., 0.]])

I can also assign a (1,3) shaped array:

In [45]: arr[:,1] = np.ones((1,3))
In [46]: arr
Out[46]: 
array([[0., 1., 0., 0., 0., 0.],
       [1., 1., 0., 0., 0., 0.],
       [2., 1., 0., 0., 0., 0.]])

But I cannot assign a (3,1) shape. That's what you are trying to do:

In [47]: arr[:,1] = np.ones((3,1))*2
Traceback (most recent call last):
  File "<ipython-input-47-d7829ffd8ee7>", line 1, in <module>
    arr[:,1] = np.ones((3,1))*2
ValueError: could not broadcast input array from shape (3,1) into shape (3,)

This is a variant on the basic broadcasting rules. When adding a (3,) to (3,1) I get a (3,3). That's because the (3,) gets promoted to (1,3) (auto add of a leading dimension):

In [48]: Out[41]+np.ones((3,1))
Out[48]: 
array([[1., 2., 3.],
       [1., 2., 3.],
       [1., 2., 3.]])

But that kind of promotion is not allowed when assigning.

Don't skip too much of the basic numpy documentation. Otherwise details like this will confuse you.