Python Difference between 2 lists

Question

I found many examples but not one with duplicate items.

I have 2 list of dict:

d1 = [{"id": 1, "value": 2}, {"id": 1, "value": 2}]
d2 = [{"id": 1, "value": 2}, {"id": 4, "value": 4}]

What I want to achieve is:

d1-d2
[{"id": 1, "value": 2}]

d2-d1
[{"id": 4, "value": 4}]

diff(d1, d2)
[{"id": 1, "value": 2}, {"id": 4, "value": 4}]

Answer 1

You can do something like this but it's not that pretty:

def sub(d1, d2):
    d2_ = d2.copy()
    res = []
    for elem in d1:
        if elem in d2_:
            d2_.remove(elem)
        else:
            res.append(elem)
    return res

Then you can use sub(d1, d2) + sub(d2, d1) to get diff(d1, d2) .

Answer 2

Assuming your values, not just the keys, in each dictionary are hashable, you can make frozenset s of each dict s .items() , collect them into a collections.Counter , and perform an xor-like operation on the Counter s to get the disparate counts. Then you just flatten out the Counter again, and convert the frozenset s back to dict s:

from collections import Counter

def diff(l1, l2):
    c1 = Counter(frozenset(d.items()) for d in l1)
    c2 = Counter(frozenset(d.items()) for d in l2)
    c1xc2 = (c1 - c2) + (c2 - c1)  # Equivalent to c1 ^ c2 if they were sets, but we need counts, and Counter doesn't support ^
    # elements() flattens the Counter to iterable of repeated values, map(dict, ...)
    # converts frozensets back to dicts
    return list(map(dict, c1xc2.elements()))

The simpler operation of computing d1 - d2 or d2 - d1 is left as an exercise; the key in all cases is converting your list of dict s to Counter s of frozenset s of the dict 's .items() , performing the operation you care about with - , then converting back to a list of dict s.

This should be fairly efficient (roughly O(n) on combined input sizes); improving on it would require a lot more custom code that's frankly not worth the bother.

Answer 3

Not elegant, but this seems to work:

def diff(d1, d2):
  found = []
  to_return = []
  for item in d1:
    if item in found:
      continue
    found += [item]
    to_return += [item] * (len(list(filter(lambda x : x == item, d1))) - len(list(filter(lambda x : x == item, d2))))
  return to_return