Loop list of dicts and combine dicts with same ID

Question

I have a list of dicts where there I need to merge multiple dicts containing the same ID key/val. What I'm doing currently is not working, it is only outputting one new dict, which is in the correct format, but I need all the merged dicts in a new list (or mutated in place, I don't much care about that).

The the list doesn't really have a min or max of how many dicts with the same ID's may present within it, it's the output of another varying function.

Here's what I have

list of dicts:

# actual ID's are longer and alphanumeric, this is for simplicity.
# dicts with same ID will also have the same 'taskConstraint', 
# but that is a side effect and can't be used as a filter
test_update_list = [
{"ID":"1","taskConstraint": "FIXT","plannedCompletionDate":"2017-07-29","constraintDate":"2017-07-29"},
{"ID":"1","taskConstraint": "FIXT","plannedStartDate":"2017-07-25","constraintDate":"2017-07-25"},
{"ID":"2","taskConstraint": "MSO","plannedCompletionDate":"2017-07-29","constraintDate":"2017-07-29"},
{"ID":"1","taskConstraint": "FIXT","noteText": "Note update text"},
{"ID":"2","taskConstraint": "MSO","noteText": "Note update text"},
{"ID":"3","taskConstraint": "MFO","noteText": "Note update text"},
{"ID":"3","taskConstraint": "MFO","plannedStartDate":"2017-07-25","constraintDate":"2017-07-25"}]

desired output:

desired_output = [
{"ID":"1","taskConstraint":"FIXT","plannedStartDate":"2017-07-25","constraintDate":"2017-07-25","plannedCompletionDate":"2017-07-29"},
{"ID":"2","taskConstraint": "MSO","plannedCompletionDate":"2017-07-29","constraintDate":"2017-07-29","noteText": "Note update text"},
{"ID":"3","taskConstraint": "MFO","plannedStartDate":"2017-07-25","constraintDate":"2017-07-25","noteText": "Note update text"}]

my awful and incorrect attempt so far:

test_update_list = [
{"ID":"1","taskConstraint": "FIXT","plannedCompletionDate":"2017-07-29","constraintDate":"2017-07-29"},
{"ID":"1","taskConstraint": "FIXT","plannedStartDate":"2017-07-25","constraintDate":"2017-07-25"},
{"ID":"2","taskConstraint": "MSO","plannedCompletionDate":"2017-07-29","constraintDate":"2017-07-29"},
{"ID":"1","taskConstraint": "FIXT","noteText": "Note update text"},
{"ID":"2","taskConstraint": "MSO","noteText": "Note update text"},
{"ID":"3","taskConstraint": "MFO","noteText": "Note update text"},
{"ID":"3","taskConstraint": "MFO","plannedStartDate":"2017-07-25","constraintDate":"2017-07-25"}]

new_update_list = []

for task in test_update_list:
    if len(new_update_list) > 0 and task not in new_update_list:
        for new_task in new_update_list:
            if task['ID'] == new_task['ID']:
                new_task = { **task, **new_task }
    else:
        new_update_list.append(task)

print(new_update_list)

which outputs...

[{'ID': '1', 'plannedCompletionDate': '2017-07-29', 'constraintDate': '2017-07-29', 'taskConstraint': 'FIXT'}]

Answer 1

You could add the new data to a dict instead of a list where the key is going to be the ID. To get the expected list of dicts call . values() on the dict later on.

>>> d = {}    
>>> for dct in test_update_list:
...     d.setdefault(dct['ID'], {}).update(dct)
...

>>> pprint(list(d.values()))
[{'ID': '1',
  'constraintDate': '2017-07-25',
  'noteText': 'Note update text',
  'plannedCompletionDate': '2017-07-29',
  'plannedStartDate': '2017-07-25',
  'taskConstraint': 'FIXT'},
 {'ID': '2',
  'constraintDate': '2017-07-29',
  'noteText': 'Note update text',
  'plannedCompletionDate': '2017-07-29',
  'taskConstraint': 'MSO'},
 {'ID': '3',
  'constraintDate': '2017-07-25',
  'noteText': 'Note update text',
  'plannedStartDate': '2017-07-25',
  'taskConstraint': 'MFO'}]