ValueError: all the input array dimensions for the concatenation axis must match exactly, but along dimension 2, the array at index 0 has size 3

Question

I am getting the error ValueError: all the input array dimensions for the concatenation axis must match exactly, but along dimension 2, the array at index 0 has size 3 and the array at index 1 has size 1 while running the below code.

for i in range(6):
    print('current b', current_batch)
   
    current_pred = model.predict(current_batch)[0]
    print('current pred', current_pred)
    
    test_predictions.append(current_pred) 
    print('current batch', current_batch)
    print('current batch => ', current_batch[:,1:,:])
    
    current_batch = np.append(current_batch[:,1:,:], [[current_pred]], axis=1)

getting this error

Can anyone please explain me why this is happening.

Thanks,

Answer 1

Basically, Numpy is telling you that the shapes of the concatenated matrices should align. For example, it is possible to concatenate a 3x4 matrix with 3x5 matrix so that we get 3x9 matrix (we added dimension 1).

The problem here is that Numpy is telling you that the axis don't align. In my example, that would be trying to concatenate 3x4 matrix with 10x10 matrix. This is not possible as the shapes are not aligned.

This usually means that the you are trying to concatenate wrong things. If you are sure though, try using np.reshape function, which will change the shape of one of the matrices so that they can be concatenated.

Answer 2

As the traceback shows, np.append is actually using np.concatenate . Did you read (study) the docs for either function? Understand what they say about dimensions?

From the display [[current_pred]] , converted to array will be (1,1,1) shape. Do you understand that?

current_batch[:,1:,:] is, as best I can tell from the small image (1,5,3)

You are asking to join on axis 1, which is 1 and 5, ok. But it's saying that the last dimension, axis 2, doesn't match. That 1 does not equal 3. Do you understand that?

List append as you do with test_predictions.append(current_pred) works well in an iteration.

np.append does not work well. Even when it works, it is slow. And here it doesn't work, because you aren't taking sufficient care to match dimensions.