[英]How to achieve numpy indexing with xarray Dataset
I know the x and the y indices of a 2D array (numpy indexing).我知道二维数组的 x 和 y 索引(numpy 索引)。
Following this documentation , xarray uses eg Fortran style of indexing.按照本文档,xarray 使用例如 Fortran 样式的索引。
So when I pass eg所以当我通过例如
ind_x = [1, 2]
ind_y = [3, 4]
I expect 2 values for the index pairs (1,3) and (2,4), but xarray returns a 2x2 matrix.我希望索引对 (1,3) 和 (2,4) 有 2 个值,但 xarray 返回一个 2x2 矩阵。
Now I want to know how to achieve numpy like indexing with xarray?现在我想知道如何像使用 xarray 索引一样实现 numpy?
Note: I want to avoid loading the whole data into memory.注意:我想避免将整个数据加载到 memory 中。 So using
.values
api is not part of the solution I am looking for.所以使用
.values
api 不是我正在寻找的解决方案的一部分。
You can access the underlying numpy
array to index it directly:您可以访问底层
numpy
数组以直接对其进行索引:
import xarray as xr
x = xr.tutorial.load_dataset("air_temperature")
ind_x = [1, 2]
ind_y = [3, 4]
print(x.air.data[0, ind_y, ind_x].shape)
# (2,)
Edit:编辑:
Assuming you have your data in a dask
-backed xarray
and don't want to load all of it into memory, you need to use vindex
on the dask
array behind the xarray
data object:假设您的数据在
dask
支持的xarray
并且不想将所有数据加载到 memory 中,您需要在xarray
数据 object 后面的dask
阵列上使用vindex
:
import xarray as xr
# simple chunk to convert to dask array
x = xr.tutorial.load_dataset("air_temperature").chunk({"time":1})
extract = x.air.data.vindex[0, ind_y, ind_x]
print(extract.shape)
# (2,)
print(extract.compute())
# [267.1, 274.1], dtype=float32)
In order to take the speed into account I have made a test with different methods.为了考虑速度,我用不同的方法进行了测试。
def method_1(file_paths: List[Path], indices) -> List[np.array]:
data=[]
for file in file_paths:
d = Dataset(file, 'r')
data.append(d.variables['hrv'][indices])
d.close()
return data
def method_2(file_paths: List[Path], indices) -> List[np.array]:
data=[]
for file in file_paths:
data.append(xarray.open_dataset(file, engine='h5netcdf').hrv.values[indices])
return data
def method_3(file_paths: List[Path], indices) -> List[np.array]:
data=[]
for file in file_paths:
data.append(xarray.open_mfdataset([file], engine='h5netcdf').hrv.data.vindex[indices].compute())
return data
In [1]: len(file_paths)
Out[1]: 4813
The results:结果:
I guess that xarray+dask takes to much time within .compute
step.我猜 xarray+dask 在
.compute
步骤中需要很多时间。
声明:本站的技术帖子网页,遵循CC BY-SA 4.0协议,如果您需要转载,请注明本站网址或者原文地址。任何问题请咨询:yoyou2525@163.com.