Fill Column using a for loop from multiple dictionaries

Question

I have multiple dictionaries and I'm trying to append them into one dataframe.

import pandas as pd
dict1 = {'A': '1', 'B': [{'att1': 'value1', 'att2': 'value2'}]}
dict2 = {'A': '2', 'B': [{'att1': 'value3', 'att2': 'value4'}]}
df = pd.DataFrame()
dict = [dict1, dict2]
df['A'] = []
for i in range(0, 2):
    df = df.append(dict[i]['B'])
    df['A'] = (dict[i]['A'])

This gives

   A    att1    att2
0  2  value1  value2
0  2  value3  value4

However I'd want

   A    att1    att2
0  1  value1  value2
0  2  value3  value4

Can anyone help please?

Answer 1

Build the row you want to append:

for i in range(0, 2):
    row = dict[i]['B']
    row[0].update({'A': dict[i]['A']})
    df = df.append(row)
    print(df)

Output:

   A    att1    att2
0  1  value1  value2

   A    att1    att2
0  1  value1  value2
0  2  value3  value4

This will likely be smoother for you if you research how to build an entire data frame from a dict (or JSON object). I've solved only the immediate problem.

Answer 2

u can use json normalize :

from pandas import json_normalize
def normalize(mapping):
    return json_normalize(mapping, 'B', 'A')

pd.concat((normalize(dict1),normalize(dict2)))
    att1    att2    A
0   value1  value2  1
0   value3  value4  2

Answer 3

You can use pd.concat

(
    pd.concat([pd.DataFrame(dict1), pd.DataFrame(dict2)])
    .pipe(lambda x: pd.concat([x,x.B.apply(pd.Series)],1))
    .drop('B',1)
)

    A   att1    att2
0   1   value1  value2
0   2   value3  value4

Answer 4

An elegant solution is possible based on pd.json_normalize , but one obstacle in this way is that B key in your both dictionaries contain a list with a single element, whereas they should contain just this element.

To overcome this obstacle, run a list comprehension, creating a list of dictionaries, with a nested dictionary comprehension converting such a list into an underlying element:

dct = [ { k: v[0] if isinstance(v, list) else v for k, v in d.items()}
    for d in [dict1, dict2]]

The result is:

[{'A': '1', 'B': {'att1': 'value1', 'att2': 'value2'}},
 {'A': '2', 'B': {'att1': 'value3', 'att2': 'value4'}}]

Then you can call df = pd.json_normalize(dct) , getting:

   A  B.att1  B.att2
0  1  value1  value2
1  2  value3  value4

And the only thing remaining to be done is to rename both B.… columns:

df.rename(columns={'B.att1' : 'att1', 'B.att2' : 'att2'}, inplace=True)

Now you have just what you want, ie:

   A    att1    att2
0  1  value1  value2
1  2  value3  value4

Caution: pd.json_normalize appeared in Pandas version 1.0 . Earlier this method was "hidden" in pandas.io.json package.

Answer 5

Moving from Allen's, I ended up getting what I need by the following:

dict1 = {'A': '1', 'B': [{'att1': 'value1', 'att2': 'value2'}]}
dict2 = {'A': '2', 'B': [{'att1': 'value3', 'att2': 'value4'}]}
dict = [dict1, dict2]

df = pd.DataFrame()
df['A'] = []
for i in range(0, 2):
    df = df.append(pd.concat([pd.DataFrame(dict[i])]).pipe(lambda x: pd.concat([x, x.B.apply(pd.Series)], 1)).drop('B', 1))
print(df)

Gives

   A    att1    att2
0  1  value1  value2
0  2  value3  value4