iterate over data frame rows in pandas

Question

Iterate over the rows in pandas and get the list of objects from rows

import pandas as pd;
df=pd.read_json("inputfile.txt")
                               data
0  {'M': {'1': 'data', '2': 'data2'}}
1    {'M': {'3': '555', '5': '3333'}}

data=[]
for row in df.iterrows():
    d = [{k1+k2:v2 for k1,v1 in x.items() for k2,v2 in v1.items()} for x in row]
    data.append(d)
print (data)

getting output like this

[[{'M1': 'data', 'M2': 'data2'}], [{'M3': '555', 'M5': '3333'}]]

need the output like this

[{'M1': 'data', 'M2': 'data2'}, {'M3': '555', 'M5': '3333'}]

Answer 1

By doing:

d = [{k1+k2:v2 for k1,v1 in x.items() for k2,v2 in v1.items()} for x in row]

You are creating a list. And appending it to data .

Modified code:

import pandas as pd

df = pd.DataFrame({'data': [{'M': {'1': 'data', '2': 'data2'}}, {'M': {'3': '555', '5': '3333'}}]})
data = []
for row in df.iterrows():
    d = {k1+k2:v2 for x in row[1] for k1,v1 in x.items() for k2,v2 in v1.items()}
    data.append(d)
print(data)

Answer 2

• .extend() - extends the list by adding all items of a list (passed as an argument) to the end.

Ex.

import pandas as pd
data = {'data':[{'M': {'1': 'data', '2': 'data2'}},{'M': {'3': '555', '5': '3333'}}]}
df = pd.DataFrame(data)
print(df)
result=[]
for row in df.iterrows():
    x = [{"{0}{1}".format(k,k1) : v1 for k,v in x[1].items() for k1,v1 in v.items()} for x in row[1].items() ]
    result.extend(x)
print(result)

Or single-line list comprehension

x = [{"{0}{1}".format(k,k1) : v1 for k,v in x[1].items() for k1,v1 in v.items()} for row in df.iterrows()
         for x in row[1].items() ]
print(x)

O/P:

                                 data
0  {'M': {'1': 'data', '2': 'data2'}}
1    {'M': {'3': '555', '5': '3333'}}


[{'M1': 'data', 'M2': 'data2'}, {'M3': '555', 'M5': '3333'}]