将 JSON 链路转换为 Pandas DataFrame

Question

Please look at the following explanation for the problem. I have a JSON Data Source: https://data.cdc.gov/api/views/x8jf-txib/rows.json and I want to convert this Data into a Pandas Data frame.

If you look at the JSON Dataset, it consists of MetaData and then the Actual Data. I would like to have a way in which I can store Metadata in a different file while the Dataset in a different file in my local System.

I have developed this method and I am not able to get it completely work for me:

from urllib.request import urlopen
import json
​
# Get the dataset
url = "https://data.cdc.gov/api/views/x8jf-txib/rows.json"
response = urlopen(url)
​
# Convert bytes to string type and string type to dict
string = response.read().decode('utf-8')
json_obj = json.loads(string)

The above Step converts the JSON File in a Dictionary and when I try to convert it into Pandas Dataframe by using this:

pd.DataFrame([json_obj.items()])

I get the output as this:

Please help me for this. I appreciate it.

Answer 1

In Python, json.loads gives you back a map/object if the JSON string was parsed properly. I think what you want to construct the DataFrame is the following:

df = pd.DataFrame.from_records(json_obj['data'])

Here's a working script:

import pandas as pd
from urllib.request import urlopen
import json

# Get the dataset
url = "https://data.cdc.gov/api/views/x8jf-txib/rows.json"
response = urlopen(url)

# Convert bytes to string type and string type to dict
string = response.read().decode('utf-8')
json_obj = json.loads(string)

df = pd.DataFrame.from_records(json_obj['data'])
print(df.head())

You should get output that looks something like:

                   0                                     1   2   ...                            38    39    40
0  row-ss5i~ibqh-im6e  00000000-0000-0000-E6C3-33C094361E41   0  ...                          None  None  None
1  row-7jrs-n8wf_crzs  00000000-0000-0000-22EC-13B75E5E7127   0  ...                          None  None  None
2  row-ddqq-yzd7.yyhz  00000000-0000-0000-319D-A1D4FB17A377   0  ...                          None  None  None
3  row-kzem-t4xs.n4ss  00000000-0000-0000-6ED5-CF3857CC1862   0  ...                          None  None  None
4  row-9ws9-2nrx~xqqg  00000000-0000-0000-3403-E46EFF15AE5B   0  ...  POINT (-89.148632 40.124144)  1721    34