如何使用numpy在for循环中向量化矩阵和？

Question

Basically I have a matrix with rows=3600 and columns=5 and wish to downsample it to parcels of 60 rows:

import numpy as np

X = np.random.rand(3600,5)

down_sample = 60
ds_rng = range(0,X.shape[0],down_sample)
X_ds = np.zeros((ds_rng.__len__(),X.shape[1]))

i = 0
for j in ds_rng:
    X_ds[i,:] = np.sum( X[j:j+down_sample,:], axis=0 )
    i += 1

Answer 1

Another way to do this might be:

def blockwise_sum(X, down_sample=60):
    n, m = X.shape

    ds_n = n / down_sample
    N = ds_n * down_sample

    if N == n:
        return np.sum(X.reshape(-1, down_sample, m), axis=1)

    X_ds = np.zeros((ds_n + 1, m))
    X_ds[:ds_n] = np.sum(X[:N].reshape(-1, down_sample, m), axis=1)
    X_ds[-1] = np.sum(X[N:], axis=0)

    return X_ds

I don't know if it's any faster though.

Answer 2

At least in this case, einsum is faster than sum .

np.einsum('ijk->ik',x.reshape(-1,down_sample,x.shape[1]))

is 2x faster than blockwise_sum .

My timings:

OP iterative  - 1.59 ms
with strided  -   198 us
blockwise_sum -   179 us
einsum        -    76 us

Answer 3

Looks like you can use some stride tricks to get the job done.

Here's the setup code we'll need:

import numpy as np
X = np.random.rand(1000,5)
down_sample = 60

And now we trick numpy into thinking X is split into parcels:

num_parcels = int(np.ceil(X.shape[0] / float(down_sample)))
X_view = np.lib.stride_tricks.as_strided(X, shape=(num_parcels,down_sample,X.shape[1]))

X_ds = X_view.sum(axis=1)  # sum over the down_sample axis

Finally, if your downsampling interval doesn't exactly divide your rows evenly, you'll need to fix up the last row in X_ds , because the stride trick we pulled made it wrap back around.

rem = X.shape[0] % down_sample
if rem != 0:
  X_ds[-1] = X[-rem:].sum(axis=0)