Collapse a nested dict into a list of dicts in python

Question

I have an input nested dict as follows:

{'name': 'Mark', 'marks':[{'english':20, 'maths':25},{'english':50, 'maths':55}]}

The output i am expecting is as follows:

[{'name': 'Mark', 'english':23, 'maths':35}, {{'name': 'Mark', 'english':50, 'maths':55}]

My code is as follows:

In [22]: input = {'name': 'Mark', 'marks':[{'english':20, 'maths':25},{'english':50, 'maths':55}]}

In [23]: marks = input.pop('marks')

In [24]: output = []

In [25]: for mark in marks:
    ...:     output.append({**input, **mark})
...:

In [26]: output
Out[26]:
[{'english': 20, 'maths': 25, 'name': 'Mark'},
 {'english': 50, 'maths': 55, 'name': 'Mark'}]

Works as expected.However, this works only for python 3.5 and above since {**x, **y} to merge 2 dicts was only introduced from that version on.

Also my dataset is huge and I am not sure if this is the most efficient way to achieve this.What is the is best way to achieve this python 2.7. I am also open to using external libraries like Pandas and numpy .

Answer 1

Here's solution using pandas :

import pandas as pd

x = {'name': 'Mark', 'marks':[{'english':20, 'maths':25},{'english':50, 'maths':55}]}

x = pd.DataFrame(x)
x = pd.concat([x['name'],x['marks'].apply(pd.Series)], axis=1)

print(x.to_dict(orient='records'))

Output:

[{'english': 20, 'name': 'Mark', 'maths': 25}, 
 {'english': 50, 'name': 'Mark', 'maths': 55}]  

PS: tested on python3 but it should work on python2.7 as well

Edit

More generic solution with additional key Now you don't have to hardcode the other keys.

x = {'name': 'Mark', 'add':'Mum','marks':[{'english':20, 'maths':25},{'english':50, 'maths':55}]}


x = pd.DataFrame(x)
cols = list(x.columns)  
cols.remove('marks')  # to get columns except `mark`

x = pd.concat([x[cols],x['marks'].apply(pd.Series)], axis=1)
print(x.to_dict(orient='records'))

Answer 2

Maybe simple list comprehension though with dirty hack to return the result:

input = {'name': 'Mark', 'marks':[{'english':20, 'maths':25},{'english':50, 'maths':55}]}

marks_list = input.pop('marks')
output = [marks.update(input) or marks for marks in marks_list]

[{'name': 'Mark', 'maths': 25, 'english': 20}, {'name': 'Mark', 'maths': 55, 'english': 50}]

Answer 3

I would use pydash ( https://pydash.readthedocs.io/en/latest/ ), it supports python 2.7:

from pydash import py_ as _
inputs = {'name': 'Mark', 'marks':[{'english':20, 'maths':25},{'english':50, 'maths':55}]}

def construct_dict(d1, d2):
   d1.update(d2)
   return d1

_(inputs['marks']).map(lambda x: construct_dict({ 'name': inputs['name'] }, x)).value()

# output
[{'maths': 25, 'name': 'Mark', 'english': 20}, {'maths': 55, 'name': 'Mark', 'english': 50}]

Answer 4

in python2.7 i would do the following:

input = {'name': 'Mark', 'marks':[{'english':20, 'maths':25},{'english':50, 'maths':55}]}

marks = input.pop('marks')
output =  map(lambda x:dict(input, **x), marks)

Answer 5

With python 2.7:

idict =  {'name': 'Mark', 'marks':[{'english':20, 'maths':25},{'english':50, 'maths':55}]}
print (idict)
marks = idict.pop("marks")
result = []
for each in marks:
    result.append({'name': idict['name']})
    for key, val in each.items():
        result[-1][key] = val
print (result)

{'name': 'Mark', 'marks': [{'maths': 25, 'english': 20}, {'maths': 55, 'english': 50}]}
[{'maths': 25, 'name': 'Mark', 'english': 20}, {'maths': 55, 'name': 'Mark', 'english': 50}]