How to make a pandas dataframe with nested columns from a list of nested dicts

Question

I have a list data containing dicts with this schema:

{'id': '123',
 '2020-06-15': {'A': 1, 'B': 2, 'C': 3, 'D': 4},
 '2020-06-16': {'A': 5, 'B': 6, 'C': 7, 'D': 8}}

When I use df = pandas.DataFrame(data).set_index("id") to create a dataframe I get this table:

                           2020-06-15                        2020-06-16
id
123  {'A': 1, 'B': 2, 'C': 3, 'D': 4}  {'A': 5, 'B': 6, 'C': 7, 'D': 8}

And when I use df = pandas.json_normalize(data) I get this:

     id  2020-06-15.A  2020-06-15.B  2020-06-15.C  2020-06-15.D  2020-06-16.A  2020-06-16.B  2020-06-16.C  2020-06-16.D
0   123             1             2             3             4             5             6             7             8 

But preferred result is this:

     2020-06-15  2020-06-16
id   A  B  C  D  A  B  C  D
123  1  2  3  4  5  6  7  8

Is there any way to make it happen?

Answer 1

Create index by id column and then use str.split :

df = df.set_index('id')
df.columns = df.columns.str.split('.', expand=True)
print (df)
    2020-06-15          2020-06-16         
             A  B  C  D          A  B  C  D
id                                         
123          1  2  3  4          5  6  7  8