无法以正确的格式将数据提取到 Pandas dataframe

Question

我正在尝试从 API 中提取数据并写入 Pandas Dataframe 以便我可以进行一些转换。

import requests

headers = {
    'Authorization': 'Api-Key',
}

params = (
    ('locodes', 'PLWRO,DEHAM'),
)

response = requests.get('https://api.xxx.com/weather/v1/forecasts', headers=headers, params=params)

API 调用的结果

response.text

'{"results":[{"place":{"type":"locode","value":"PLWRO"},"measures":[{"ts":1571896800000,"t2m":10.72,"t_min":10.53,"t_max":11.99,"wspd":8,"dir":"SE","wgust":12,"rh2m":87,"prsmsl":1012,"skcover":"clear","precip":0.0,"snowd":0,"thunderstorm":"N","fog":"H"}]},{"place":{"type":"locode","value":"DEHAM"},"measures":[{"ts":1571896800000,"t2m":10.79,"t_min":10.3,"t_max":10.9,"wspd":13,"dir":"ESE","wgust":31,"rh2m":97,"prsmsl":1008,"skcover":"partly_cloudy","precip":0.0,"snowd":0,"thunderstorm":"N","fog":"H"}]}]}'

当尝试进入 pandas dataframe 时，它的格式不正确。

import pandas as pd
import io
urlData = response.content
rawData = pd.read_csv(io.StringIO(urlData.decode('utf-8')))

当前Output

如何在每个 header 下正确填充值。

预期格式

Answer 1

首先将 json 转换为字典，然后需要进行一些处理以将locode添加到 measure 、合并字典、 measures列表并最后调用DataFrame构造函数：

import json

d = json.loads(response.text)

out = []
for x in d['results']:
    t = x['place']['type']
    v = x['place']['value']
    for y in x['measures']:
        y = {**{t:v}, **y}
        out.append(y)
#print (out)

df = pd.DataFrame(out)
print (df)
  locode             ts    t2m  t_min  t_max  wspd  dir  wgust  rh2m  prsmsl  \
0  PLWRO  1571896800000  10.72  10.53  11.99     8   SE     12    87    1012   
1  DEHAM  1571896800000  10.79  10.30  10.90    13  ESE     31    97    1008   

         skcover  precip  snowd thunderstorm fog  
0          clear     0.0      0            N   H  
1  partly_cloudy     0.0      0            N   H  

Answer 2

您可以处理使用模块抽象语法树（导入 ast）将字符串转换为 Python 字典。 您可以在此StackOverflow 帖子中阅读有关 ast 用户案例的更多信息

在你的情况下，我会这样做：

import ast
import pandas as pd

response = '{"results":[{"place":{"type":"locode","value":"PLWRO"},"measures":[{"ts":1571896800000,"t2m":10.72,"t_min":10.53,"t_max":11.99,"wspd":8,"dir":"SE","wgust":12,"rh2m":87,"prsmsl":1012,"skcover":"clear","precip":0.0,"snowd":0,"thunderstorm":"N","fog":"H"}]},{"place":{"type":"locode","value":"DEHAM"},"measures":[{"ts":1571896800000,"t2m":10.79,"t_min":10.3,"t_max":10.9,"wspd":13,"dir":"ESE","wgust":31,"rh2m":97,"prsmsl":1008,"skcover":"partly_cloudy","precip":0.0,"snowd":0,"thunderstorm":"N","fog":"H"}]}]}'

# convert response to python dict
response_to_dict = ast.literal_eval(response)

# convert response_to_dict into pandas DataFrame
df = pd.DataFrame(response_to_dict['results'][0]['measures'])

Output：

|---|---|------|------|----|---|-----|------------|----|-----|----|
|dir|fog|precip|prsmsl|rh2m|...|t_min|thunderstorm|ts  |wgust|wspd|
|SE |H  |0.0   |1012  | 87 |...|10.53|N           |15..|12   | 8  |
|---|---|------|------|----|---|-----|------------|----|-----|----|