[英]Numpy Vectorization of sliding-window operation
I have the following numpy arrays: 我有以下numpy数组:
arr_1 = [[1,2],[3,4],[5,6]] # 3 X 2
arr_2 = [[0.5,0.6],[0.7,0.8],[0.9,1.0],[1.1,1.2],[1.3,1.4]] # 5 X 2
arr_1
is clearly a 3 X 2
array, whereas arr_2
is a 5 X 2
array. arr_1
显然是3 X 2
阵列,而arr_2
是5 X 2
阵列。
Now without looping, I want to element-wise multiply arr_1 and arr_2 so that I apply a sliding window technique (window size 3) to arr_2. 现在没有循环,我想以元素方式乘以arr_1和arr_2,以便我将滑动窗口技术(窗口大小3)应用于arr_2。
Example:
Multiplication 1: np.multiply(arr_1,arr_2[:3,:])
Multiplication 2: np.multiply(arr_1,arr_2[1:4,:])
Multiplication 3: np.multiply(arr_1,arr_2[2:5,:])
I want to do this in some sort of a matrix multiplication form to make it faster than my current solution which is of the form: 我想以某种矩阵乘法形式做到这一点,使它比我目前的解决方案更快,形式如下:
for i in (2):
np.multiply(arr_1,arr_2[i:i+3,:])
So if the number of rows in arr_2 are large (of the order of tens of thousands), this solution doesn't really scale very well. 因此,如果arr_2中的行数很大(数万个数量级),那么这个解决方案并不能很好地扩展。
Any help would be much appreciated. 任何帮助将非常感激。
We can use NumPy broadcasting
to create those sliding windowed indices in a vectorized manner. 我们可以使用
NumPy broadcasting
以矢量化方式创建那些滑动窗口索引。 Then, we can simply index into arr_2
with those to create a 3D
array and perform element-wise multiplication with 2D
array arr_1
, which in turn will bring on broadcasting
again. 然后,我们可以简单地将
arr_2
索引到那些用于创建3D
数组并使用2D
数组arr_1
执行元素乘法的arr_1
,这反过来将再次引入broadcasting
。
So, we would have a vectorized implementation like so - 所以,我们会有一个像这样的矢量化实现 -
W = arr_1.shape[0] # Window size
idx = np.arange(arr_2.shape[0]-W+1)[:,None] + np.arange(W)
out = arr_1*arr_2[idx]
Runtime test and verify results - 运行时测试并验证结果 -
In [143]: # Input arrays
...: arr_1 = np.random.rand(3,2)
...: arr_2 = np.random.rand(10000,2)
...:
...: def org_app(arr_1,arr_2):
...: W = arr_1.shape[0] # Window size
...: L = arr_2.shape[0]-W+1
...: out = np.empty((L,W,arr_1.shape[1]))
...: for i in range(L):
...: out[i] = np.multiply(arr_1,arr_2[i:i+W,:])
...: return out
...:
...: def vectorized_app(arr_1,arr_2):
...: W = arr_1.shape[0] # Window size
...: idx = np.arange(arr_2.shape[0]-W+1)[:,None] + np.arange(W)
...: return arr_1*arr_2[idx]
...:
In [144]: np.allclose(org_app(arr_1,arr_2),vectorized_app(arr_1,arr_2))
Out[144]: True
In [145]: %timeit org_app(arr_1,arr_2)
10 loops, best of 3: 47.3 ms per loop
In [146]: %timeit vectorized_app(arr_1,arr_2)
1000 loops, best of 3: 1.21 ms per loop
This is a nice case to test the speed of as_strided
and Divakar's broadcasting. 这是测试
as_strided
和Divakar广播速度的一个很好的例子。
In [281]: %%timeit
...: out=np.empty((L,W,arr1.shape[1]))
...: for i in range(L):
...: out[i]=np.multiply(arr1,arr2[i:i+W,:])
...:
10 loops, best of 3: 48.9 ms per loop
In [282]: %%timeit
...: idx=np.arange(L)[:,None]+np.arange(W)
...: out=arr1*arr2[idx]
...:
100 loops, best of 3: 2.18 ms per loop
In [283]: %%timeit
...: arr3=as_strided(arr2, shape=(L,W,2), strides=(16,16,8))
...: out=arr1*arr3
...:
1000 loops, best of 3: 805 µs per loop
Create Numpy array without enumerating array for more of a comparison of these methods. 创建Numpy数组而不枚举数组,以便更多地比较这些方法。
声明:本站的技术帖子网页,遵循CC BY-SA 4.0协议,如果您需要转载,请注明本站网址或者原文地址。任何问题请咨询:yoyou2525@163.com.