Efficient conversion of a 3D numpy array to a 1D numpy array

Question

I have a 3D numpy array in this form:

>>>img.shape
(4504932, 2, 2)

>>> img
array([[[15114, 15306],
    [15305, 15304]],

   [[15305, 15306],
    [15303, 15304]],

   [[15305, 15306],
    [15303, 15304]],

   ..., 

   [[15305, 15302],
    [15305, 15302]]], dtype=uint16)

Which I want to convert to a 1D numpy array where each entry is the sum of each 2x2 submatrix in the above img numpy array.

I have been able to accomplish this using:

img_new = np.array([i.sum() for i in img])
>>> img_new
array([61029, 61218, 61218, ..., 61214, 61214, 61214], dtype=uint64)

Which is exactly what I want. But this is too slow (takes about 10 seconds). Is there a faster method I could use? I included above img.shape so you had an idea of the size of this numpy array.

EDIT - ADDITIONAL INFO: My img array could also be a 3D array in the form of 4x4, 5x5, 7x7.. etc submatrices. This is specified by the variables sub_rows and sub_cols .

Answer 1

img.sum(axis=(1, 2))

sum allows you to specify an axis or axes along which to sum, rather than just summing the whole array. This allows NumPy to loop over the array in C and perform just a few machine instructions per sum, rather than having to go through the Python bytecode evaluation loop and create a ton of wrapper objects to stick in a list.

Answer 2

Using a numpy method ( apply_over_axes ) is usually quicker and indeed that is the case here. I just tested on a 4000x2x2 array:

img = np.random.rand(4000,2,2)
timeit(np.apply_along_axis(np.sum, img, [1,2]))
# 1000 loops, best of 3: 721 us per loop
timeit(np.array([i.sum() for i in img]))
# 100 loops, best of 3: 17.2 ms per loop

Answer 3

You can use np.einsum -

img_new = np.einsum('ijk->i',img)

Verify results

In [42]: np.array_equal(np.array([i.sum() for i in img]),np.einsum('ijk->i',img))
Out[42]: True

Runtime tests

In [34]: img = np.random.randint(0,10000,(10000,2,2)).astype('uint16')

In [35]: %timeit np.array([i.sum() for i in img]) # Original approach
10 loops, best of 3: 92.4 ms per loop

In [36]: %timeit img.sum(axis=(1, 2)) # From other solution
1000 loops, best of 3: 297 µs per loop

In [37]: %timeit np.einsum('ijk->i',img)
10000 loops, best of 3: 102 µs per loop