convert list of dicts to pandas dataframe

Question

This is my dataset

I have an odd one that I have been working on for a week and need some help now. I have a list of dicts like this:

[index 0 : {'Total_Salary': 49900.0, 'Total_Value': 490.0, 
 'pers_1': {'value': 71.1, 'Name': 'Bob', 'Salary': 10100, 'nick_name': 'foo'}, 
 'pers_2': {'value': 43.1, 'Name': 'Joe', 'Salary': 9200, 'nick_name': 'bar'}}
 'pers_3': {'value': 42.1, 'Name': 'james', 'Salary': 9750, 'nick_name': 'sam'}}
 'pers_4': {'value': 41.1, 'Name': 'rick', 'Salary': 9700, 'nick_name': 'suzy'}}
 'pers_5': {'value': 23.1, 'Name': 'blop', 'Salary': 9400, 'nick_name': 'jill'}}
 'pers_6': {'value': 54.1, 'Name': 'burp', 'Salary': 9280, 'nick_name': 'yup'}}

index 1: (will have a different total salary, total value number because the people will change, but the format stays the same as above) ... index n: 'Total_Salary' = ..., 'Total_Value' = ..., person_n... ]

Every dictionary in the list of dicts has a total_salary and total_value key. It is the sum of each persons 'Salary' and 'Value' front the person-1 through person-6. The list has several hundred dicts just like above appended. I want to loop through the list of dicts and put each dict into one dataframe.

Ideally, the dataframe would be indexed with Team 1 as an index (then team 2 , team 3 , etc).

Answer 1

import pandas as pd

team = {
    'pers_1': {'value': 71.1, 'Name': 'Bob', 'Salary': 10100, 'nick_name': 'foo'}, 
    'pers_2': {'value': 43.1, 'Name': 'Joe', 'Salary': 9200, 'nick_name': 'bar'},
    'pers_3': {'value': 42.1, 'Name': 'james', 'Salary': 9750, 'nick_name': 'sam'},
    'pers_4': {'value': 41.1, 'Name': 'rick', 'Salary': 9700, 'nick_name': 'suzy'},
    'pers_5': {'value': 23.1, 'Name': 'blop', 'Salary': 9400, 'nick_name': 'jill'},
    'pers_6': {'value': 54.1, 'Name': 'burp', 'Salary': 9280, 'nick_name': 'yup'}}

df = (pd.DataFrame(team)
      .T
      .append(pd.Series({'value': df['value'].sum(),
                         'Salary': df['Salary'].sum()},
                        name='Total'))
      .assign(team='team_1')
      .set_index('team', append=True)
      .swaplevel())

print(df)

Results in

                Name  Salary nick_name  value
team                                         
team_1 pers_1    Bob   10100       foo   71.1
       pers_2    Joe    9200       bar   43.1
       pers_3  james    9750       sam   42.1
       pers_4   rick    9700      suzy   41.1
       pers_5   blop    9400      jill   23.1
       pers_6   burp    9280       yup   54.1
       Total     NaN  172290       NaN  823.8

You can do the same for the other teams, and then concatenate all the dataframes. In mixed pseudocode:

all_teams_list = []
# loop for all teams
    # create `df`
    all_teams_list.append(df)
all_teams = pd.concat(all_teams_list)