在数据框中转换嵌套字典？

Question

I have been trying to parse the nested dict in data frame. I made this df from dict, but couldn't figure out this nested one.

df

    First   second    third              

 0     1       2      {nested dict}

nested dict:

   {'fourth': '4', 'fifth': '5', 'sixth': '6'}, {'fourth': '7', 'fifth': '8', 'sixth': '9'}

My Desired output would be:

        First   second  fourth   fifth   sixth   fourth   fifth   sixth          

 0     1       2       4         5        6         7       8       9

Edit: original Dict

   'archi': [{'fourth': '115',
      'fifth': '-162',
      'sixth': '112'},
     {'fourth': '52',
      'fifth': '42',
      'sixth': ' 32'}]

Answer 1

I can't quit tell the format of the nested dict in the "third" column, but here is what I recommend using Python: Pandas dataframe from Series of dict as a starting point. Here is a dict and dataframe which are reproducible:

nst_dict = {'archi': [{'fourth': '115', 'fifth': '-162', 'sixth': '112'},
      {'fourth': '52', 'fifth': '42','sixth': ' 32'}]}

df = pd.DataFrame.from_dict({'First':[1,2], 'Second':[2,3], 
     'third': [nst_dict,nst_dict]})

Then you need to first access the list within the dict, then the items of the list:

df.thrd_1 = df.third.apply(lambda x: x['archi']) # convert to list
df.thrd_1a = df.thrd_1.apply(lambda x: x[0]) # access first item
df.thrd_1b = df.thrd_1.apply(lambda x: x[1]) # access second item

out = df.drop('third', axis=1).merge(
    df.thrd_1a.apply(pd.Series).merge(df.thrd_1a.apply(pd.Series),
    left_index=True, right_index=True),
    left_index=True, right_index=True)

print(out)

First  Second fourth_x fifth_x sixth_x fourth_y fifth_y sixth_y
0      1       2      115    -162     112      115    -162     112
1      2       3      115    -162     112      115    -162     112

I will try to clean this up with collections.abc and turn into a function, but this should do the trick for your specific case.

Answer 2

A "brute force" approach

import pandas as pd
import numpy as np

my_dict = {'Zero': 0, 'First': 1, 'Second': 2,
       'archi': [{'fourth': '115', 'fifth': '-162', 'sixth': '112'},
                {'fourth': '52', 'fifth': '42', 'sixth': ' 32'}]}

data_row=[]
columns = []
for key in my_dict.keys():
    try:
        if len(my_dict[key]):
            for item in my_dict[key]:
                # iterate over nested dicts
                for k, v in item.items():
                    columns.append(k)
                    data_row.append(v)

    except TypeError:
        data_row.append(my_dict[key])
        columns.append(key)

print(columns)
print(data_row)

data = np.array(data_row).reshape(1,9)
df = pd.DataFrame(new_d, columns=columns)
print(df)

Output:

     Zero   First   Second   fourth     fifth   sixth   fourth  fifth   sixth
0       0       1        2      115      -162     112      52      42      32

Answer 3

I created a function using a recursive approach to flatten the dict structure:

original_dict = {'Zero': 0, 'First': 1, 'Second': 2,
       'archi': [{'fourth': '115', 'fifth': '-162', 'sixth': '112'},
                {'fourth': '52', 'fifth': '42', 'sixth': ' 32'}]}

flattened_dict = {}

def flatten(obj, name = ''):
    if isinstance(obj, dict):
        for key, value in obj.items():
            flatten(obj[key], key)
    elif isinstance(obj, list):
        for e in obj:
            flatten(e)
    else:
        flattened_dict[name] = [obj] 

flatten(original_dict)

Then the creation of the dataframe:

pd.DataFrame(flattened_dict)

With the following output: