[英]How to efficiently concatenate many arange calls in numpy?
我想在cnt
值的向量上向numpy.arange(0, cnt_i)
這樣的調用進行向量化,並像這個片段一樣連接結果:
import numpy
cnts = [1,2,3]
numpy.concatenate([numpy.arange(cnt) for cnt in cnts])
array([0, 0, 1, 0, 1, 2])
不幸的是,由於臨時數組和列表推導循環,上面的代碼非常低效。
有沒有辦法在numpy中更有效地做到這一點?
這是一個完全矢量化的函數:
def multirange(counts):
counts = np.asarray(counts)
# Remove the following line if counts is always strictly positive.
counts = counts[counts != 0]
counts1 = counts[:-1]
reset_index = np.cumsum(counts1)
incr = np.ones(counts.sum(), dtype=int)
incr[0] = 0
incr[reset_index] = 1 - counts1
# Reuse the incr array for the final result.
incr.cumsum(out=incr)
return incr
這里的@開發者回答的變化,只有調用arange
一次:
def multirange_loop(counts):
counts = np.asarray(counts)
ranges = np.empty(counts.sum(), dtype=int)
seq = np.arange(counts.max())
starts = np.zeros(len(counts), dtype=int)
starts[1:] = np.cumsum(counts[:-1])
for start, count in zip(starts, counts):
ranges[start:start + count] = seq[:count]
return ranges
這是原始版本,作為函數編寫:
def multirange_original(counts):
ranges = np.concatenate([np.arange(count) for count in counts])
return ranges
演示:
In [296]: multirange_original([1,2,3])
Out[296]: array([0, 0, 1, 0, 1, 2])
In [297]: multirange_loop([1,2,3])
Out[297]: array([0, 0, 1, 0, 1, 2])
In [298]: multirange([1,2,3])
Out[298]: array([0, 0, 1, 0, 1, 2])
使用更多的計數比較時間:
In [299]: counts = np.random.randint(1, 50, size=50)
In [300]: %timeit multirange_original(counts)
10000 loops, best of 3: 114 µs per loop
In [301]: %timeit multirange_loop(counts)
10000 loops, best of 3: 76.2 µs per loop
In [302]: %timeit multirange(counts)
10000 loops, best of 3: 26.4 µs per loop
嘗試以下解決內存問題,效率幾乎相同。
out = np.empty((sum(cnts)))
k = 0
for cnt in cnts:
out[k:k+cnt] = np.arange(cnt)
k += cnt
所以沒有使用連接。
np.tril_indices
幾乎為你做了這個:
In [28]: def f(c):
....: return np.tril_indices(c, -1)[1]
In [29]: f(10)
Out[29]:
array([0, 0, 1, 0, 1, 2, 0, 1, 2, 3, 0, 1, 2, 3, 4, 0, 1, 2, 3, 4, 5, 0, 1,
2, 3, 4, 5, 6, 0, 1, 2, 3, 4, 5, 6, 7, 0, 1, 2, 3, 4, 5, 6, 7, 8])
In [33]: %timeit multirange(range(10))
10000 loops, best of 3: 93.2 us per loop
In [34]: %timeit f(10)
10000 loops, best of 3: 68.5 us per loop
當維度很小時,比@Warren Weckesser multirange
快得多。
但是當尺寸更大時變得慢得多(@hpaulj,你有一個很好的觀點):
In [36]: %timeit multirange(range(1000))
100 loops, best of 3: 5.62 ms per loop
In [37]: %timeit f(1000)
10 loops, best of 3: 68.6 ms per loop
聲明:本站的技術帖子網頁,遵循CC BY-SA 4.0協議,如果您需要轉載,請注明本站網址或者原文地址。任何問題請咨詢:yoyou2525@163.com.