Read NetCDF file in python

Question

I am trying to read a NetCDF file from IRI/LDEO Climate Data Library (dust_pm25_sconc10_mon) but I am with a problem to read this file. When I select the variables that compuse the database (longitude (X), latitude (Y) and time (T)), the output from X and Y are a sequence with the number of observations (1, 2, ..., 139 for example). That is, the values of longitude and latitude are not exported corretly.

Someone could help me with this problem? I already tried read this file with R, Python and Qgis and in all of these threes the output of X and Y are the same.

My codes are below (Python).

Thank you all very much.

from netCDF4 import Dataset as dt

filestr = 'dust_pm25_sconc10_mon.nc'

ncfile = dt(filestr, 'r')

print(ncfile.variables)

lat = ncfile.variables['Y'][:]
lat

lon = ncfile.variables['X'][:]
lon

time = ncfile.variables['T'][:]
time

Answer 1

Edit:

This file has three independent variables, X, Y, and T. And the values of X and Y intentionally go from 1 to len(X) and len(Y) respectively.

Look at the description of the file: http://iridl.ldeo.columbia.edu/home/.nasa_roses_a19/.Dust_model/.dust_mon_avg/.dust_pm25_sconc10_mon/

Independent Variables (Grids)
Time
grid: /T (months since 1960-01-01) ordered (Mar 1979) to (Mar 2010) by 1.0 N= 373 pts :grid
Longitude
grid: /X (unitless) ordered (1.0) to (191.0) by 1.0 N= 191 pts :grid
Latitude
grid: /Y (unitless) ordered (1.0) to (139.0) by 1.0 N= 139 pts :grid

Of course, this might be meaningful for longitude, but for latitude this is nonsense. Unfortunately, I did not find any hint which area on this planet this dataset should describe.

However, I also did not find any data in it's only dependent variable dust_pm25_sconc10_mon - it's empty.

PS: Just as an example:
This dataset here http://iridl.ldeo.columbia.edu/home/.nasa_roses_a19/.Dust_model/.RegDustModelProjected/.dust_pm25_sconc10/datafiles.html
looks much more reasonable...

The description alone is much more promising:

Independent Variables (Grids)
Time (time)
grid: /T (days since 2009-01-02 00:00) ordered (0130-0430 2 Jan 2009) to (2230 1 Apr 2010 - 0130 2 Apr 2010) by 0.125 N= 3640 pts :grid
Longitude
grid: /X (degree_east) ordered (19.6875W) to (54.6875E) by 0.625 N= 120 pts :grid
Latitude
grid: /Y (degree_north) ordered (0.3125N) to (39.6875N) by 0.625 N= 64 pts :grid

And its dependent variable dust_pm25_sconc10 is also not empty.

---

I really tried to find this file on the website you mentioned, but it is futile imo. So without knowing it, I have to guess:

netcdf-files provide the possibility to save data space by scaling and shifting the values of any variable so that they can be stored eg as int instead of float .
You could simply check, if there are attributes add_offset other than 0 and scale_factor other than 1.

For further information about this concept you can refer to https://www.unidata.ucar.edu/software/netcdf/workshops/2010/bestpractices/Packing.html .

While the information in the link above states that the java interface to netcdf does apply these attributes automatically, the netcdf4-python library does not. So if you want to stay with this package, you have to rescale and -offset the data back to the original values as described.

However, you could also consider trying out xarray , a library which implements the n-dimensional datastructure of netcdf files and as far ss I experienced, this library does automatic scaling and offsetting according to the rules described above.
http://xarray.pydata.org/en/stable/

Answer 2

The example file at http://iridl.ldeo.columbia.edu/home/.nasa_roses_a19/.Dust_model/.dust_mon_avg/.dust_pm25_sconc10_mon/datafiles.html that you linked in your comment on SpghttCd's response is not well-formed. For one thing, the X and Y arrays do not have units attributes appropriate to such dimensions but instead both have value "units". And as already noted the values in the arrays don't "look" valid anyway. Further, the values in the dust_pm25_sconc10_mon array in that file all appear to be NaN.

On the other hand the example dataset at http://iridl.ldeo.columbia.edu/home/.nasa_roses_a19/.Dust_model/.RegDustModelProjected/.dust_pm25_sconc10/datafiles.html that SpghttCd references has good units attribute information ("degrees_east" and "degrees_north", respectively). Furthermore, the actual values in the X and Y arrays look good. I had no problem making a plot of the dust_pm25_sconc10 variable in that dataset (using Panoply) and seeing the data mapped over the appropriate region.

SpghttCd's comments regarding scaling and offsets do not apply here as the longitude and latitudes in that second, good file have actual lon and lat values.