如何将列表多嵌套字典转换为 pandas DataFrame 并转换 DataFrame？

Question

I need to convert a list of nested dictionaries (game_stats) to a pandas DataFrame. I have tried to do so with "games_stats_df" but I am getting a DataFrame with "id" and then a list of dictionaries again.

print(game_stats)

[{'id': 401282099,
 'teams': [{'conference': 'SEC',
            'homeAway': 'away',
            'points': 21,
            'school': 'LSU',
            'stats': [{'category': 'rushingTDs', 'stat': '2'},
                      {'category': 'passingTDs', 'stat': '1'},
                      {'category': 'kickingPoints', 'stat': '3'},
                      {'category': 'fumblesRecovered', 'stat': '0'},
                      {'category': 'firstDowns', 'stat': '22'}]},
           {'conference': 'SEC',
            'homeAway': 'home',
            'points': 42,
            'school': 'Kentucky',
            'stats': [{'category': 'rushingTDs', 'stat': '3'},
                      {'category': 'passingTDs', 'stat': '4'},
                      {'category': 'kickingPoints', 'stat': '0'},
                      {'category': 'fumblesRecovered', 'stat': '1'},
                      {'category': 'firstDowns', 'stat': '24'}]}]}]

game_stats_df = pd.DataFrame.from_records([game.to_dict() for game in game_stats])
print(game_stats_df.head())

          id                                              teams
0  401282099  [{'school': 'LSU', 'conference': 'SEC', 'homeA...

Ideally, I am trying to get a DataFrame with the below format:

game_id    school   conference  homeAway    points  rushingTDs  passingTDs  etc
401282099   LSU      SEC          away       21        2            1

Answer 1

Use json_normalize :

df = pd.json_normalize(game_stats, 'teams', 'id').explode('stats').reset_index()
df = pd.concat([df, pd.json_normalize(df.pop('stats'))], axis=1)
df = df.pivot_table('stat', df.columns[1:-2].tolist(), 'category').reset_index()

Output:

>>> df
category conference homeAway  points    school         id  firstDowns  fumblesRecovered  kickingPoints  passingTDs  rushingTDs
0               SEC     away      21       LSU  401282099        22.0               0.0            3.0         1.0         2.0
1               SEC     home      42  Kentucky  401282099        24.0               1.0            0.0         4.0         3.0