How to create a dict of dicts from pandas dataframe?

Question

I have a dataframe df

id      price      date         zipcode
u734    8923944    2017-01-05   AERIU87
uh72    9084582    2017-07-28   BJDHEU3
u029    299433     2017-09-31   038ZJKE

I want to create a dictionary with the following structure

{'id': xxx, 'data': {'price': xxx, 'date': xxx, 'zipcode': xxx}}

What I have done so far

ids = df['id']
prices = df['price']
dates = df['date']
zips = df['zipcode']
d = {'id':idx, 'data':{'price':p, 'date':d, 'zipcode':z} for idx,p,d,z in zip(ids,prices,dates,zips)}
>>> SyntaxError: invalid syntax

but I get the error above.

What would be the correct way to do this, using either

• list comprehension

OR

• pandas .to_dict()

bonus points: what is the complexity of the algorithm, and is there a more efficient way to do this?

Answer 1

I'd suggest the list comprehension.

v = df.pop('id')
data = [
   {'id' : i, 'data' : j} 
   for i, j in zip(v, df.to_dict(orient='records'))
]

Or a compact version,

data = [dict(id=i, data=j) for i, j in zip(df.pop('id'), df.to_dict(orient='r'))]

Note that, if you're popping id inside the expression, it has to be the first argument to zip .

print(data)
[{'data': {'date': '2017-09-31',
   'price': 299433,
   'zipcode': '038ZJKE'},
  'id': 'u029'},
 {'data': {'date': '2017-01-05',
   'price': 8923944,
   'zipcode': 'AERIU87'},
  'id': 'u734'},
 {'data': {'date': '2017-07-28',
   'price': 9084582,
   'zipcode': 'BJDHEU3'},
  'id': 'uh72'}]