python list of dictionaries find duplicates based on value

Question

I have a list of dicts:

a =[{'id': 1,'desc': 'smth'},
    {'id': 2,'desc': 'smthelse'},
    {'id': 1,'desc': 'smthelse2'},
    {'id': 1,'desc': 'smthelse3'}]

I would like to go trough the list and find those dicts that have the same id value (eg id=1 ) and create a new dict:

b = [{'id':1, 'desc' : [smth, smthelse2,smthelse3]}, 
     {'id': 2, 'desc': 'smthelse'}]

Answer 1

You can try:

import operator, itertools

key = operator.itemgetter('id')

b = [{'id': x, 'desc': [d['desc'] for d in y]} 
     for x, y in itertools.groupby(sorted(a, key=key), key=key)]

Answer 2

It is better to keep the "desc" values as lists everywhere even if they contain a single element only. This way you can do

for d in b:
    print d['id']
    for desc in d['desc']:
        print desc

This would work for strings too, just returning individual characters, which is not what you want.

And now the solution giving you a list of dicts of lists:

a =[{'id': 1,'desc': 'smth'},{'id': 2,'desc': 'smthelse'},{'id': 1,'desc': 'smthelse2'},{'id': 1,'desc': 'smthelse3'}]

c = {}
for d in a:
    c.setdefault(d['id'], []).append(d['desc'])
b = [{'id': k, 'desc': v} for k,v in c.iteritems()]

b is now:

[{'desc': ['smth', 'smthelse2', 'smthelse3'], 'id': 1},
 {'desc': ['smthelse'], 'id': 2}]

Answer 3

from collections import defaultdict

d = defaultdict(list)
for x in a:
    d[x['id']].append(x['desc']) # group description by id
b = [dict(id=id, desc=desc if len(desc) > 1 else desc[0])
     for id, desc in d.items()]

To preserve order:

b = []
for id in (x['id'] for x in a):
    desc = d[id]
    if desc:
       b.append(dict(id=id, desc=desc if len(desc) > 1 else desc[0]))
       del d[id]