从字典词典列表创建Pandas数据框

Question

I have a list of dictionaries, where each dictionary represents a record. It is formatted as follows:

>>> ListOfData=[
... {'Name':'Andrew',
...  'number':4,
...  'contactinfo':{'Phone':'555-5555', 'Address':'123 Main St'}},
... {'Name':'Ben',
...  'number':6,
...  'contactinfo':{'Phone':'555-5554', 'Address':'124 2nd St'}},
... {'Name':'Cathy',
...  'number':1,
...  'contactinfo':{'Phone':'555-5556', 'Address':'126 3rd St'}}]
>>> 
>>> import pprint
>>> pprint.pprint(ListOfData)
[{'Name': 'Andrew',
  'contactinfo': {'Address': '123 Main St', 'Phone': '555-5555'},
  'number': 4},
 {'Name': 'Ben',
  'contactinfo': {'Address': '124 2nd St', 'Phone': '555-5554'},
  'number': 6},
 {'Name': 'Cathy',
  'contactinfo': {'Address': '126 3rd St', 'Phone': '555-5556'},
  'number': 1}]
>>> 

What is the best way to read this into a Pandas dataframe with multiindex columns for those attributes in the sub dictionaries?

For example, I'd ideally have 'Phone' and 'Address' columns nested under the 'contactinfo' columns.

I can read in the data as follows, but would like the contact info column to be broken into sub columns.

>>> pd.DataFrame.from_dict(ListOfData)
     Name                                        contactinfo  number
0  Andrew  {u'Phone': u'555-5555', u'Address': u'123 Main...       4
1     Ben  {u'Phone': u'555-5554', u'Address': u'124 2nd ...       6
2   Cathy  {u'Phone': u'555-5556', u'Address': u'126 3rd ...       1
>>> 

Answer 1

how about this

declare empty data frame

df = DataFrame(columns=('Name', 'conntactinfo', 'number'))

then iterate over List and add rows

for row in ListOfData:
    df.loc[len(df)] = row

complete code

import pandas as pd

ListOfData=[
 {'Name':'Andrew',
  'number':4,
  'contactinfo':{'Phone':'555-5555', 'Address':'123 Main St'}},
 {'Name':'Ben',
  'number':6,
  'contactinfo':{'Phone':'555-5554', 'Address':'124 2nd St'}}]

df = pd.DataFrame(columns=('Name', 'contactinfo', 'number'))

for row in ListOfData:

    df.loc[len(df)] = row

print(df)

this prints

  Name                                      contactinfo  number
0  Andrew  {'Phone': '555-5555', 'Address': '123 Main St'}       4
1     Ben   {'Phone': '555-5554', 'Address': '124 2nd St'}       6

Answer 2

Here is a pretty clunky workaround that I was able to get what I need. I loop through the columns, find those that are made of dicts and then divide it into multiple columns and merge it to the dataframe. I'd appreciate hearing any ways to improve this code. I'd imagine that ideally the dataframe would be constructed from the get-go without having dictionaries as values.

>>> df=pd.DataFrame.from_dict(ListOfData)
>>> 
>>> for name,col in df.iteritems():
...     if any(isinstance(x, dict) for x in col.tolist()):
...         DividedDict=col.apply(pd.Series)
...         DividedDict.columns=pd.MultiIndex.from_tuples([(name,x) for x in DividedDict.columns.tolist()])
...         df=df.join(DividedDict)
...         df.drop(name,1, inplace=True)
... 
>>> print df
     Name  number (contactinfo, Address) (contactinfo, Phone)
0  Andrew       4            123 Main St             555-5555
1     Ben       6             124 2nd St             555-5554
2   Cathy       1             126 3rd St             555-5556
>>> 

Answer 3

Don't know about best or not, but you could do it in two steps:

>>> df = pd.DataFrame(ListOfData)
>>> df = df.join(pd.DataFrame.from_records(df.pop("contactinfo")))
>>> df
     Name  number      Address     Phone
0  Andrew       4  123 Main St  555-5555
1     Ben       6   124 2nd St  555-5554
2   Cathy       1   126 3rd St  555-5556