Noaa Api with Python. Downloaded the datasets, how will I open them?

Question

I have tried to access datasets from NOAA, for a project. I have been able to download the json file but I do not know how to open the desired file I have printed out.

url = "http://www.ncdc.noaa.gov/cdo-web/api/v2/data?datasetid=GHCNDMS"
headers = {"token":"my token"}
response = json.loads(requests.get(url, "dataset", headers = headers).text)
response = response['results']
response = response[0]
print(response)

This is my output:

{'uid': 'gov.noaa.ncdc:C00861', 'mindate': '1763-01-01', 'maxdate': '2018-04-22', 'name': 'Daily Summaries', 'datacoverage': 1, 'id': 'GHCND'}

How can I access the data inside this data set as I wish to turn them into a Pandas DataFrame

Answer 1

Your output is a dictionary, which can be directly turned into a pandas dataframe by df = pd.DataFrame([response]) - NOTICE , I am passing this dictionary as a list to the pandas DataFrame constructor.

Or you can tailor your url to get the desired information. In the example below, I am getting data from the specified dataset and date ranges which are easily gleaned/edited.

It seems that NOAA has updated their api formats at some time, and a lot of the material out there is not quite working anymore, so others may be receiving errors with this example code. If you are just getting started, this format will return a json file with results, and hopefully you can adapt it from there:

import requests
import pandas as pd
import json
from datetime import datetime

token = 'yourtoken'

url = "http://www.ncdc.noaa.gov/cdo-web/api/v2/data?datasetid=GHCNDMS&startdate=1776-07-04&enddate=1776-09-04"
headers = {"token":token}

r = requests.get(url, "dataset", headers = headers).text

response = json.loads(r)
response = response['results']
response = response[0]
print(response)

df=pd.DataFrame([response])