使用setdefault重构Python字典列表

Question

I am trying to re-structure a list of Python dictionaries by 'grouping' (that's probably not the correct expression, but using it as a proxy based on SQL) the dictionaries based on a (non-unique) value. I have got close, however I'm falling at the final hurdle, in that I couldn't work out how to re-assign each value to a name (ie I end up with what looks like a tuple rather than a dict).

Additionally I have a superfluous list (ie my output ends up as [[{...}]] rather than [{...}].

I have used the example here:

How do I group this list of dicts by the same month?

Which gets me quite close to what I want, however I'm stuck at the final stage!

market = [
    {'selection_id': 1099, 'value': '11', 'value_name': 'a'},
    {'selection_id': 1099, 'value': '78', 'value_name': 'p'},
    {'selection_id': 1097, 'value': '39', 'value_name': 'b'},
    {'selection_id': 1097, 'value': '52', 'value_name': 'f'},
    {'selection_id': 1098, 'value': '98', 'value_name': 'd'},
    {'selection_id': 1099, 'value': '13', 'value_name': 'y'},
    {'selection_id': 1098, 'value': '4', 'value_name': 'r'},
]

new_structure = {}
new_structure2 = []

for z in market:
        new_structure.setdefault(z['selection_id'], []).append((z['value'], z['value_name']))
        t = [{'selection_id': m, 'value_dict': n} for m, n in new_structure.items()]
new_structure2.append(t)

print(new_structure2)

This outputs as:

[[{'selection_id': 1099, 'value_dict': [('11', 'a'), ('78', 'p'), ('13',  
 'y')]}, {'selection_id': 1097, 'value_dict': [('39', 'b'), ('52', 'f')]},  
 {'selection_id': 1098, 'value_dict': [('98', 'd'), ('4', 'r')]}]]

Which is very close, however what I'm aiming for is:

[{'selection_id': 1099,
  'value_dict': [{'value': '11', 'value_name': 'a'},
                 {'value': '78', 'value_name': 'p'},
                 {'value': '13', 'value_name': 'y'}]},
 {'selection_id': 1097,
  'value_dict': [{'value': '39', 'value_name': 'b'},
                 {'value': '52', 'value_name': 'f'}]},
 {'selection_id': 1098,
  'value_dict': [{'value': '98', 'value_name': 'd'},
                 {'value': '4', 'value_name': 'r'}]}]

I appreciate its probably a really simple fix, but its escaping me at the moment, so any guidance would be greatly appreciated!

Answer 1

Here are some hints:

The first thing is to sort by selection_id :

by_selection_id = operator.itemgetter('selection_id')
market.sort(key=by_selection_id)

Then you can group by selection_id :

for selection_id, group in itertools.groupby(market, key=by_selection_id):
    print(selection_id, list(group))

You get:

(1097, [{'value_name': 'b', 'value': '39', 'selection_id': 1097},
        {'value_name': 'f', 'value': '52', 'selection_id': 1097}])
(1098, [{'value_name': 'd', 'value': '98', 'selection_id': 1098},
        {'value_name': 'r', 'value': '4', 'selection_id': 1098}])
(1099, [{'value_name': 'a', 'value': '11', 'selection_id': 1099},
        {'value_name': 'p', 'value': '78', 'selection_id': 1099},
        {'value_name': 'y', 'value': '13', 'selection_id': 1099}])

Then it is easy to build the final list.

Here is a solution using comprehension list/dict:

new_structure = [{'selection_id': selection_id,
                  'value_dict': [{'value': item['value'],
                                  'value_name': item['value_name']} for item in group]}
                 for selection_id, group in itertools.groupby(market, key=by_selection_id)]

Or using a classic list with append :

new_structure = []
for selection_id, group in itertools.groupby(market, key=by_selection_id):
    value_dict = [{'value': item['value'], 'value_name': item['value_name']} for item in group]
    new_structure.append({'selection_id': selection_id,
                          'value_dict': value_dict})

Answer 2

I end up with what looks like a tuple rather than a dict)

This is because you are appending a tuple, not a dict:

.append((z['value'], z['value_name']))

Answer 3

So, just to update, based on the answer from @Code-Apprentice, I have re-written my code as follows:

market = [
    {'selection_id': 1099, 'value': '11', 'value_name': 'a'},
    {'selection_id': 1099, 'value': '78', 'value_name': 'p'},
    {'selection_id': 1097, 'value': '39', 'value_name': 'b'},
    {'selection_id': 1097, 'value': '52', 'value_name': 'f'},
    {'selection_id': 1098, 'value': '98', 'value_name': 'd'},
    {'selection_id': 1099, 'value': '13', 'value_name': 'y'},
    {'selection_id': 1098, 'value': '4', 'value_name': 'r'},
]

new_structure = {}
new_structure2 = []

for z in market:
    new_structure.setdefault(z['selection_id'], []).append({'value': z['value'],
                                                            'value_name': z['value_name']})
    t = [{'selection_id': m, 'value_dict': n} for m, n in new_structure.items()]
    new_structure2.append(t)

print(new_structure2)

Which then gets me very close to my desired output. The only remaining issue I have, is a superfluous list (ie [[{....}]] instead of [{...}]), I believe this is probably because of the way I have written the line t=....

Here is my output, I will post an update, once I've got rid of that extra list in case the answer is of use to anyone else (but please feel free to shout if you spot it before me):

[[{'selection_id': 1099,
   'value_dict': [{'value': '11', 'value_name': 'a'},
                  {'value': '78', 'value_name': 'p'},
                  {'value': '13', 'value_name': 'y'}]},
  {'selection_id': 1097,
   'value_dict': [{'value': '39', 'value_name': 'b'},
                  {'value': '52', 'value_name': 'f'}]},
  {'selection_id': 1098,
   'value_dict': [{'value': '98', 'value_name': 'd'},
                  {'value': '4', 'value_name': 'r'}]}]]

So ridiculously basic! I was over-complicating it, just simply removed the extra list reference:

market = [
    {'selection_id': 1099, 'value': '11', 'value_name': 'a'},
    {'selection_id': 1099, 'value': '78', 'value_name': 'p'},
    {'selection_id': 1097, 'value': '39', 'value_name': 'b'},
    {'selection_id': 1097, 'value': '52', 'value_name': 'f'},
    {'selection_id': 1098, 'value': '98', 'value_name': 'd'},
    {'selection_id': 1099, 'value': '13', 'value_name': 'y'},
    {'selection_id': 1098, 'value': '4', 'value_name': 'r'},
]

new_structure = {}

for z in market:
    new_structure.setdefault(z['selection_id'], []).append({'value': z['value'],
                                                            'value_name': z['value_name']})

new_structure2 = [{'selection_id': m, 'value_dict': n} for m, n in new_structure.items()]