[英]Improving performance of numpy mapping operation
I have a numpy array of size (4, X, Y), where the first dimension stands for an (R,G,B,A) quadruplet. 我有一个大小不等的数组(4,X,Y),其中第一个维度代表(R,G,B,A)四联体。 My aim is to transpose each X*Y
RGBA quadruplets to X*Y
floating-point values, given a dictionary matching them. 我的目标是将每个X*Y
RGBA四元组转换为X*Y
浮点值,给定与它们匹配的字典。
My current code is as follows: 我目前的代码如下:
codeTable = {
(255, 255, 255, 127): 5.5,
(128, 128, 128, 255): 6.5,
(0 , 0 , 0 , 0 ): 7.5,
}
for i in range(0, rows):
for j in range(0, cols):
new_data[i,j] = codeTable.get(tuple(data[:,i,j]), -9999)
Where data
is a numpy array of size (4, rows, cols)
, and new_data
is of size (rows, cols)
. data
是一个大小不一的数组(4, rows, cols)
, new_data
的大小(rows, cols)
。
The code is working fine, but takes quite a long time. 代码工作正常,但需要很长时间。 How should I optimize that piece of code? 我该如何优化这段代码?
Here is a full example: 这是一个完整的例子:
import numpy
codeTable = {
(253, 254, 255, 127): 5.5,
(128, 129, 130, 255): 6.5,
(0 , 0 , 0 , 0 ): 7.5,
}
# test data
rows = 2
cols = 2
data = numpy.array([
[[253, 0], [128, 0], [128, 0]],
[[254, 0], [129, 144], [129, 0]],
[[255, 0], [130, 243], [130, 5]],
[[127, 0], [255, 120], [255, 5]],
])
new_data = numpy.zeros((rows,cols), numpy.float32)
for i in range(0, rows):
for j in range(0, cols):
new_data[i,j] = codeTable.get(tuple(data[:,i,j]), -9999)
# expected result for `new_data`:
# array([[ 5.50000000e+00, 7.50000000e+00],
# [ 6.50000000e+00, -9.99900000e+03],
# [ 6.50000000e+00, -9.99900000e+03], dtype=float32)
Here's an approach that returns your expected result, but with such a small amount of data it's hard to know if this will be faster for you. 这是一种返回预期结果的方法,但由于数据量很少,很难知道这对您来说是否更快。 Since I've avoided the double for loop, however, I imagine you'll see a pretty decent speedup. 因为我已经避免了双循环,但我想你会看到相当不错的加速。
import numpy
import pandas as pd
codeTable = {
(253, 254, 255, 127): 5.5,
(128, 129, 130, 255): 6.5,
(0 , 0 , 0 , 0 ): 7.5,
}
# test data
rows = 3
cols = 2
data = numpy.array([
[[253, 0], [128, 0], [128, 0]],
[[254, 0], [129, 144], [129, 0]],
[[255, 0], [130, 243], [130, 5]],
[[127, 0], [255, 120], [255, 5]],
])
new_data = numpy.zeros((rows,cols), numpy.float32)
for i in range(0, rows):
for j in range(0, cols):
new_data[i,j] = codeTable.get(tuple(data[:,i,j]), -9999)
def create_output(data):
# Reshape your two data sources to be a bit more sane
reshaped_data = data.reshape((4, -1))
df = pd.DataFrame(reshaped_data).T
reshaped_codeTable = []
for key in codeTable.keys():
reshaped = list(key) + [codeTable[key]]
reshaped_codeTable.append(reshaped)
ct = pd.DataFrame(reshaped_codeTable)
# Merge on the data, replace missing merges with -9999
result = df.merge(ct, how='left')
newest_data = result[4].fillna(-9999)
# Reshape
output = newest_data.reshape(rows, cols)
return output
output = create_output(data)
print(output)
# array([[ 5.50000000e+00, 7.50000000e+00],
# [ 6.50000000e+00, -9.99900000e+03],
# [ 6.50000000e+00, -9.99900000e+03])
print(numpy.array_equal(new_data, output))
# True
The numpy_indexed package (disclaimer: I am its author) contains a vectorized nd-array capable variant of list.index, which can be used to solve your problem efficiently and concisely: numpy_indexed包(免责声明:我是它的作者)包含一个vector.ndex的矢量化nd数组变体,它可以用来有效和简洁地解决你的问题:
import numpy_indexed as npi
map_keys = np.array(list(codeTable.keys()))
map_values = np.array(list(codeTable.values()))
indices = npi.indices(map_keys, data.reshape(4, -1).T, missing='mask')
remapped = np.where(indices.mask, -9999, map_values[indices.data]).reshape(data.shape[1:])
声明:本站的技术帖子网页,遵循CC BY-SA 4.0协议,如果您需要转载,请注明本站网址或者原文地址。任何问题请咨询:yoyou2525@163.com.