How can I parse a DataFrame column where each row contains a Dict, into independent DataFrame columns?

Question

I have the following DataFrame:

    TradeDictionary
0   {'id': '545', 'instrument': 'EUR_USD', 'price'...
1   {'id': '539', 'instrument': 'GBP_USD', 'price'...
2   {'id': '535', 'instrument': 'EUR_USD', 'price'...
3   {'id': '529', 'instrument': 'EUR_USD', 'price'...
4   {'id': '523', 'instrument': 'EUR_USD', 'price'...
5   {'id': '501', 'instrument': 'EUR_USD', 'price'...
6   {'id': '495', 'instrument': 'GBP_USD', 'price'...
7   {'id': '489', 'instrument': 'EUR_USD', 'price'...
8   {'id': '483', 'instrument': 'EUR_USD', 'price'...
9   {'id': '477', 'instrument': 'EUR_USD', 'price'...
10  {'id': '471', 'instrument': 'GBP_USD', 'price'...

However I need to convert it into separate columns in a DataFrame. I've tried numerous suggestions in other posts but doesn't seem to like it.

Answer 1

Use pd.Series in df.apply :

In [1426]: df = pd.DataFrame({'TradeDictionary':[{'id': '545', 'instrument': 'EUR_USD', 'price':100}, {'id': '539', 'instrument': 'GBP_USD', 'price':200}, {'id': '535', 'instrument': 'EUR_USD', 'price':50}]})

In [1427]: df
Out[1427]: 
                                     TradeDictionary
0  {'id': '545', 'instrument': 'EUR_USD', 'price'...
1  {'id': '539', 'instrument': 'GBP_USD', 'price'...
2  {'id': '535', 'instrument': 'EUR_USD', 'price'...

In [1429]: df1 = df.TradeDictionary.apply(pd.Series)

In [1430]: df1
Out[1430]: 
    id instrument  price
0  545    EUR_USD    100
1  539    GBP_USD    200
2  535    EUR_USD     50

OR for better performance , use pd.json_normalize :

In [1432]: df1 = pd.json_normalize(df.TradeDictionary)