简体   繁体   中英

How can I more efficiently extract data from netCDF files in python?

I have written the following code to extract data from several netCDF files. I have 192 files of each time step of half hour (ie 4 days of data). I have considered 10 latitude and 10 longitude values (ie 100 data points).

In the output I am getting the 192 time steps in the row and data for different points in column.

The output is what I wanted, but I think the code is not efficient.

import glob
from netCDF4 import Dataset
import pandas as pd
import numpy as np

#Record all the years of the netCDF files into a python list

all_hh = []   # ##all_years is a python list
date_range = pd.date_range(start = str(2000)+'-12-30',
                            end = str(2001)+'-01-03',
                            freq ='30 min')
d_range_mod = date_range.drop(pd.Timestamp("2001-01-03T00:00:00.000000000"))
lng = range(0,10,1)
ltd = range(0,10,1)

intn = []
for file in glob.glob('*.nc4'):
    # print(file)
    data = Dataset(file, 'r')
    all_hh.append(file)
    
for i in all_hh:
    data = Dataset(i,'r')
    temp = data.variables['precipitationCal']    
    for x in lng:
        for y in ltd:
            inten = temp[0,x,y]
            intn.append(inten)

df1 = pd.DataFrame(intn, columns=['Intensity (mm/hr)'])
df2 = np.array(df1)
df3 = np.reshape(df2, (192,100))
df4 = pd.DataFrame(df3, index = d_range_mod)
    
df4.to_excel('intensity_tS.xlsx')    

This can be done more easily with xarray:

import xarray as xr

(
xr.open_mfdataset(glob.glob('*.nc4'))
.to_dataframe()
.to_excel('intensity_tS.xlsx')
      )

With some modifications, obviously, depending on precisely what is in your data files.

The technical post webpages of this site follow the CC BY-SA 4.0 protocol. If you need to reprint, please indicate the site URL or the original address.Any question please contact:yoyou2525@163.com.

 
粤ICP备18138465号  © 2020-2024 STACKOOM.COM