無法以正確的格式將數據提取到 Pandas dataframe

Question

我正在嘗試從 API 中提取數據並寫入 Pandas Dataframe 以便我可以進行一些轉換。

import requests

headers = {
    'Authorization': 'Api-Key',
}

params = (
    ('locodes', 'PLWRO,DEHAM'),
)

response = requests.get('https://api.xxx.com/weather/v1/forecasts', headers=headers, params=params)

API 調用的結果

response.text

'{"results":[{"place":{"type":"locode","value":"PLWRO"},"measures":[{"ts":1571896800000,"t2m":10.72,"t_min":10.53,"t_max":11.99,"wspd":8,"dir":"SE","wgust":12,"rh2m":87,"prsmsl":1012,"skcover":"clear","precip":0.0,"snowd":0,"thunderstorm":"N","fog":"H"}]},{"place":{"type":"locode","value":"DEHAM"},"measures":[{"ts":1571896800000,"t2m":10.79,"t_min":10.3,"t_max":10.9,"wspd":13,"dir":"ESE","wgust":31,"rh2m":97,"prsmsl":1008,"skcover":"partly_cloudy","precip":0.0,"snowd":0,"thunderstorm":"N","fog":"H"}]}]}'

當嘗試進入 pandas dataframe 時，它的格式不正確。

import pandas as pd
import io
urlData = response.content
rawData = pd.read_csv(io.StringIO(urlData.decode('utf-8')))

當前Output

如何在每個 header 下正確填充值。

預期格式

Answer 1

首先將 json 轉換為字典，然后需要進行一些處理以將locode添加到 measure 、合並字典、 measures列表並最后調用DataFrame構造函數：

import json

d = json.loads(response.text)

out = []
for x in d['results']:
    t = x['place']['type']
    v = x['place']['value']
    for y in x['measures']:
        y = {**{t:v}, **y}
        out.append(y)
#print (out)

df = pd.DataFrame(out)
print (df)
  locode             ts    t2m  t_min  t_max  wspd  dir  wgust  rh2m  prsmsl  \
0  PLWRO  1571896800000  10.72  10.53  11.99     8   SE     12    87    1012   
1  DEHAM  1571896800000  10.79  10.30  10.90    13  ESE     31    97    1008   

         skcover  precip  snowd thunderstorm fog  
0          clear     0.0      0            N   H  
1  partly_cloudy     0.0      0            N   H  

Answer 2

您可以處理使用模塊抽象語法樹（導入 ast）將字符串轉換為 Python 字典。 您可以在此StackOverflow 帖子中閱讀有關 ast 用戶案例的更多信息

在你的情況下，我會這樣做：

import ast
import pandas as pd

response = '{"results":[{"place":{"type":"locode","value":"PLWRO"},"measures":[{"ts":1571896800000,"t2m":10.72,"t_min":10.53,"t_max":11.99,"wspd":8,"dir":"SE","wgust":12,"rh2m":87,"prsmsl":1012,"skcover":"clear","precip":0.0,"snowd":0,"thunderstorm":"N","fog":"H"}]},{"place":{"type":"locode","value":"DEHAM"},"measures":[{"ts":1571896800000,"t2m":10.79,"t_min":10.3,"t_max":10.9,"wspd":13,"dir":"ESE","wgust":31,"rh2m":97,"prsmsl":1008,"skcover":"partly_cloudy","precip":0.0,"snowd":0,"thunderstorm":"N","fog":"H"}]}]}'

# convert response to python dict
response_to_dict = ast.literal_eval(response)

# convert response_to_dict into pandas DataFrame
df = pd.DataFrame(response_to_dict['results'][0]['measures'])

Output：

|---|---|------|------|----|---|-----|------------|----|-----|----|
|dir|fog|precip|prsmsl|rh2m|...|t_min|thunderstorm|ts  |wgust|wspd|
|SE |H  |0.0   |1012  | 87 |...|10.53|N           |15..|12   | 8  |
|---|---|------|------|----|---|-----|------------|----|-----|----|