Python Download NetCDF file from a website which provides the file after clicking a button

Question

If you go to this website: https://ruc.noaa.gov/raobs/Data_request.cgi?byr=2010&bmo=5&bdy=26&bhr=12&eyr=2010&emo=5&edy=27&ehr=15&shour=All+Times&ltype=All+Levels&wunits=Knots&access=WMO+Station+Identifier

Type "72632" into the box, and change "Format" to "NetCDF format (binary)", and then click "Continue Data Access", a NetCDF file is downloaded to your computer.

If I use the Chrome developer tools to track network activity after clicking this button, I can see the the "Request URL" which leads to this file being downloaded is: https://ruc.noaa.gov/raobs/GetRaobs.cgi?shour=All+Times&ltype=All+Levels&wunits=Knots&bdate=2010052612&edate=2010052715&access=WMO+Station+Identifier&view=NO&StationIDs=72632&osort=Station+Series+Sort&oformat=NetCDF+format+%28Binary%29

If you copy and paste that URL into a web browser, the file is downloaded.

What I want to do is use Python to take a URL formatted like the one above, and retrieve the associated NetCDF file.

I've had luck in the past doing something like

url = 'https://ruc.noaa.gov/raobs/GetRaobs.cgi?shour=All+Times&ltype=All+Levels&wunits=Knots&bdate=2010052612&edate=2010052715&access=WMO+Station+Identifier&view=NO&StationIDs=72632&osort=Station+Series+Sort&oformat=NetCDF+format+%28Binary%29'
da  = xr.open_dataset(url)

But that doesn't work in this case:

OSError: [Errno -75] NetCDF: Malformed or unexpected Constraint: b'https://ruc.noaa.gov/raobs/GetRaobs.cgi?shour=All+Times&ltype=All+Levels&wunits=Knots&bdate=2010052612&edate=2010052715&access=WMO+Station+Identifier&view=NO&StationIDs=72632&osort=Station+Series+Sort&oformat=NetCDF+format+%28Binary%29'

I've also tried to wget the URL, but that just downloads a ".cgi" file which I don't think is useful.

Thanks for any help!

Answer 1

You could use my package nctoolkit to download the file and then export to xarray. This will save the file to a temporary directory, but will remove it once the session is done.

import nctoolkit as nc
import xarray as xr
ds = nc.open_url("https://ruc.noaa.gov/raobs/GetRaobs.cgi?shour=All+Times&ltype=All+Levels&wunits=Knots&bdate=2010052612&edate=2010052715&access=WMO+Station+Identifier&view=NO&StationIDs=72632&osort=Station+Series+Sort&oformat=NetCDF+format+%28Binary%29")
ds_xr = ds.to_xarray()