Convert Nested Dictionary into Dataframe(Pandas)

Question

I have nested dictionary as below. I want to create Dataframe as shown in expected output .

Code-

dct = { "a" : 
              { "b" : 
                    [{ "2022-03-31" : { "c":"20"} },
                     { "2021-03-31" : { "c" : "30" } } ]  },
        "e" : { "b" : 
                    [{ "2022-03-31" : { "d":"300"} },
                     { "2021-03-31" : { "d" : "29" } } ] } ,
        "h" : 
              { "b" : 
                    [ {"2022-03-31" : { "f":"190"} },
                     { "2021-03-31" : { "f" : "98" } } ] } }


hi = pd.DataFrame.from_dict(dct, orient='index')   
hi

I tried above code but it's not giving desired output.

Expected Output-

date           c     d      f    
2022-03-31    20    30    300
2021-03-31    30    29     98

Answer 1

To flatten the dict, so the dates become the index:

date_indexed_dict = {}
for sub_dict in dct.values():
    for list_item in sub_dict['b']:
        for date, col_dict in list_item.items():
            if date not in date_indexed_dict:
                date_indexed_dict[date] = {}
            # add the columns as keys on each index (date)
            for col, val in col_dict.items():
                date_indexed_dict[date][col] = val
print(date_indexed_dict)
# {'2022-03-31': {'c': '20', 'd': '300', 'f': '190'},
# '2021-03-31': {'c': '30', 'd': '29', 'f': '98'}}

df = pd.DataFrame.from_dict(date_indexed_dict, orient='index')
print(df)