Hi I want speed up my calculation which is very slow using this ndnumerate loop:
The loop below goes through a numpy array and carrys out some math on each cell but ignores the -9999 values, keeping them the same.
my_array = np.array([[-9999, 1, 1],
[2, 2, -9999],
[3, 3, 3]])
# Intialise two empty arrays
1_d = np.empty_like(my_array, dtype = float)
3_d = np.empty_like(my_array, dtype = float)
start = time.time()
for [x, y], value in np.ndenumerate(my_array):
if value >= 0:
1_d[x, y] = value - (20 * (100 - value)) / ((100 - value) + math.exp(2.533 - 0.0636 * (100 - value)))
3_d[x, y] = value * math.exp(0.00673 * (100 - value))
else:
1_d[x, y] = -9999
3_d[x, y] = -9999
print "Calculating numbers took " + str(round(time.time() - start,2)) + "s.")
You shouldn't do this with a loop, but just use the vectorized nature of numpy arrays, as this is perfectly possible in this case:
a1_d = my_array - (20 * (100 - my_array)) / ((100 - my_array) + np.exp(2.533 - 0.0636 * (100 - my_array)))
a3_d = my_array * np.exp(0.00673 * (100 - my_array))
To have the -9999 values back, you can do:
a1_d[my_array == -9999] = -9999
a3_d[my_array == -9999] = -9999
or another option would be to use np.nan
instead of -9999, which would just propagate:
my_array = my_array.astype(float)
my_array[my_array == -9999] = np.nan
or another option would be to do boolean indexing during the calculation:
valid = (my_array != -9999)
a1_d[valid] = my_array[valid] * ...
a3_d[~valid] = -9999
For this small example array, this takes ca 70 µs instead of 260 µs with the for loop (using %%timeit
)
You can use masked array
:
import numpy as np
my_array = np.array([[-9999, 1, 1],
[2, 2, -9999],
[3, 3, 3]])
value = np.ma.masked_values(my_array, -9999)
d1 = value - (20 * (100 - value)) / ((100 - value) + np.exp(2.533 - 0.0636 * (100 - value)))
d3 = value * np.exp(0.00673 * (100 - value))
The technical post webpages of this site follow the CC BY-SA 4.0 protocol. If you need to reprint, please indicate the site URL or the original address.Any question please contact:yoyou2525@163.com.