[英]Convert Nested JSON to Pandas Dataframe (with JSON example)
I have a JSON blob which looks like this:我有一个 JSON blob,如下所示:
{'status': 'OK',
'data-availability': 'available',
'data': [{'page': 1, 'pages': 1, 'total': 7},
[{'domain_id': '101',
'domain_name': 'Province1',
'domain_url': 'https://province1.com'},
{'domain_id': '102',
'domain_name': 'Province2',
'domain_url': 'https://province2.com'},
{'domain_id': '103',
'domain_name': 'Province3',
'domain_url': 'https://province3.com'},
{'domain_id': '104',
'domain_name': 'Province4',
'domain_url': 'https://province4.com'},
{'domain_id': '105',
'domain_name': 'Province5',
'domain_url': 'https://province5.com'},
{'domain_id': '106',
'domain_name': 'Province6',
'domain_url': 'https://province6.com'},
{'domain_id': '107',
'domain_name': 'Province7',
'domain_url': 'https://province7.com'}]]}
What I want is to normalize it into Pandas DataFrame which column are consist of domain_id, domain_name, and domain_url.我想要的是将其规范化为 Pandas DataFrame 哪些列由 domain_id、domain_name 和 domain_url 组成。
How can I accomplish this?我怎样才能做到这一点?
Repeated appending to a dataframe is slow .重复追加一个 dataframe 很慢。 Instead, collect everything in a dictionary and then call
.from_dict()
:相反,将所有内容收集到字典中,然后调用
.from_dict()
:
from pandas import pd
result = defaultdict(list)
for entry in data['data'][1]:
for key, value in entry.items():
result[key].append(value)
print(pd.DataFrame.from_dict(result))
This outputs:这输出:
domain_id domain_name domain_url
0 101 Province1 https://province1.com
1 102 Province2 https://province2.com
2 103 Province3 https://province3.com
3 104 Province4 https://province4.com
4 105 Province5 https://province5.com
5 106 Province6 https://province6.com
6 107 Province7 https://province7.com
This does the job,这完成了工作,
data = json.loads(test)["data"][-1]
df = pd.DataFrame()
for d in data:
temp_df = pd.DataFrame([data[0]])
df = pd.concat([df, temp_df])
You can use pd.json_normalize() .您可以使用pd.json_normalize() 。
raw_data = [{'domain_id': '101',
'domain_name': 'Province1',
'domain_url': 'https://province1.com'},
{'domain_id': '102',
'domain_name': 'Province2',
'domain_url': 'https://province2.com'},
{'domain_id': '103',
'domain_name': 'Province3',
'domain_url': 'https://province3.com'},
{'domain_id': '104',
'domain_name': 'Province4',
'domain_url': 'https://province4.com'},
{'domain_id': '105',
'domain_name': 'Province5',
'domain_url': 'https://province5.com'},
{'domain_id': '106',
'domain_name': 'Province6',
'domain_url': 'https://province6.com'},
{'domain_id': '107',
'domain_name': 'Province7',
'domain_url': 'https://province7.com'}]
# store data as df
df = pd.DataFrame({'raw':raw_data})
# split dict into columns with keys as column names
df_json = pd.json_normalize(df['raw'])
# concat dfs
df = pd.concat([df, df_json], axis=1)
# display
display(df)
声明:本站的技术帖子网页,遵循CC BY-SA 4.0协议,如果您需要转载,请注明本站网址或者原文地址。任何问题请咨询:yoyou2525@163.com.