Trouble reshaping 3-d NumPy array into 2-d NumPy array

Question

I'm working on a problem with image processing, and my data is presented as a 3-dimensional NumPy array, where the (x, y, z) entry is the (x, y) pixel (numerical intensity value) of image z. There are 100000 images and each image is 25x25. Thus, the data matrix is of size 25x25x10000. I am trying to convert this into a 2-dimensional matrix of size 10000x625, where each row is a linearization of the pixels in the image. For example, suppose that instead the images were 3x3, we would have the following:

1 2 3
4 5 6  ------> [1, 2, 3, 4, 5, 6, 7, 8, 9]
7 8 9

I am attempting to do this by calling data.reshape((10000, 625)) , but the data is no longer aligned properly after doing so. I have tried transposing the matrix in valid stages of reshaping, but that does not seem to fix it.

Does anyone know how to fix this?

Answer 1

If you want the data to be aligned you need to do data.reshape((625, 10000)) .

If you want a different layout try np.rollaxis :

data_rolled = np.rollaxis(data, 2, 0) # This is Shape (10000, 25, 25)
data_reshaped = data_rolled.reshape(10000, 625) # Now you can do your reshape.

Numpy needs you to know which elements belong together during reshaping, so only "merge" dimensions that belong together.

Answer 2

The problem is that you aren't respecting the standard index order in your reshape call. The data will only be aligned if the two dimensions you want to combine are in the same position in the new array ( (25, 25, 10000) -> (625, 10000) ).

Then, to get the shape you want, you can transpose. It's easier to visualize with a smaller example -- when you run into problems like this, always try out a smaller example in the REPL if you can.

>>> a = numpy.arange(12)
>>> a = a.reshape(2, 2, 3)
>>> a
array([[[ 0,  1,  2],
        [ 3,  4,  5]],

       [[ 6,  7,  8],
        [ 9, 10, 11]]])
>>> a.reshape(4, 3)
array([[ 0,  1,  2],
       [ 3,  4,  5],
       [ 6,  7,  8],
       [ 9, 10, 11]])
>>> a.reshape(4, 3).T
array([[ 0,  3,  6,  9],
       [ 1,  4,  7, 10],
       [ 2,  5,  8, 11]])

No need to rollaxis !

Notice how the print layout that numpy uses makes this kind of reasoning easier. The differences between the first and the second step are only in the bracket positions; the numbers all stay in the same place, which often helps when you want to think through shape issues.