How to remove a json string from list in python without losing data

Question

My question is similar to that of another question in SO How to remove a json string from list . The solution to that question did solve a part of my problem but mine is little different.

My lists are:

list1 = [{"ID": 1, "data": "12"},{"ID": 2, "data": "13"}]
list2 = [{"ID": 1, "col": "5"},{"ID": 1, "col": "8"},{"ID": 2,"col": "2"}]

I did the following to modify the final list:

per_id = {}
for info in chain(list1, list2):
    per_id.setdefault(info['ID'], {}).update(info)

output = list(per_id.values())

The expected output was:

output = [{"ID": 1,"data": "12", "col": "5"},{"ID": 1,"data": "12", "col": "8"},{"ID": 2,"data": "13","col": "2"}]

But the output i got is:

output = [{"ID": 1,"data": "12", "col": "5"},{"ID": 2,"data": "13","col": "2"}]

Is there a way to rectify this problem.

Answer 1

You get the second version because there is more than one "ID" with the the value of 1. If you use a defaultdict with a list, you can set it to append.

This example is taken directly from the page:

>>> s = [('yellow', 1), ('blue', 2), ('yellow', 3), ('blue', 4), ('red', 1)]
>>> d = defaultdict(list)
>>> for k, v in s:
...     d[k].append(v)
...
>>> d.items()
[('blue', [2, 4]), ('red', [1]), ('yellow', [1, 3])]

Answer 2

If you have one list that contains some attributes for an ID and another one that contains other attributes, chaining is probably not the best idea.

Here you could simply iterate separately both lists and update the map of one list with the map of the second one. Code example in python console:

>>> result = []  # start with an empty list 
>>> for elt1 in list1:
        for elt2 in list2:
            if elt1['ID'] == elt2['ID']:
                eltr = elt1.copy()  # take a copy to avoid changing original lists
                eltr.update(elt2)
                result.append(eltr)

>>> result
[{'data': '12', 'ID': 1, 'col': '5'}, {'data': '12', 'ID': 1, 'col': '8'}, {'data': '13', 'ID': 2, 'col': '2'}]

as expected...

Answer 3

Try itertools-combinations-

from itertools import chain
from itertools import combinations

list1 = [{"ID": 1, "data": "12"},{"ID": 2, "data": "13"}]
list2 = [{"ID": 1, "col": "5"},{"ID": 1, "col": "8"},{"ID": 2,"col": "2"}]


data = []
for i,j in combinations(chain(list1,list2),2):
    if i['ID'] == j['ID']:
        d = dict(i.items()+j.items())
        if len(d.keys())==3:#Ensure that it has three keys i.e. avoid combination between lsit1 elements or combination between list2 elements themselves.
            data.append(d)
print data

Output-

[{'data': '12', 'ID': 1, 'col': '5'}, {'data': '12', 'ID': 1, 'col': '8'}, {'data': '13', 'ID': 2, 'col': '2'}]

Answer 4

The previous answer from Serge Ballesta works if you include a simple check to avoid repeating values (i would comment it, but i don't have enough reputation).

result = [] # start with an empty list

for elt1 in list1:
    for elt2 in list2:
        if elt1['ID'] == elt2['ID']:
            for k in elt2.keys():
                if k != "ID":
                    eltr = elt1.copy()  # take a copy to avoid changing original lists
                    eltr.update(elt2)
                    result.append(eltr)

result

Output:

[{'data': '12', 'ID': 1, 'col': '5'}, {'data': '12', 'ID': 1, 'col': '8'}, {'data': '13', 'ID': 2, 'col': '2'}]