Merging a list of dictionaries based on common key values but with nested dict of lists

Question

Trying to wrap my head around (what I feel) is a way too complicated dict/list to represent the relationship between items.

mappings = [{'switches': ['switch1', 'switch2'], 'relationships': [{'compute_name': 'compute001', 'servers': ['server1', 'server2']}]},
            {'switches': ['switch1', 'switch2'], 'relationships': [{'compute_name': 'compute002', 'servers': ['server3', 'server4']}]},
            {'switches': ['switch3', 'switch4'], 'relationships': [{'compute_name': 'compute003', 'servers': ['server5', 'server6']}]},
            {'switches': ['switch3', 'switch4'], 'relationships': [{'compute_name': 'compute004', 'servers': ['server7', 'server8', 'server9']}]},
            {'switches': ['switch5', 'switch6'], 'relationships': [{'compute_name': 'compute005', 'servers': ['server10', 'server11', 'server12']}]},
            {'switches': ['switch5', 'switch6'], 'relationships': [{'compute_name': 'compute006', 'servers': ['server13', 'server14', 'server15', 'server16']}]}]

As you can see, it's a list of dict, where the value is sometimes another list of dict.

The issue I'm having is that the starting data are holding "duplicates". For example, if you look at the key value of "switches", you can see that the list holding "switch1" + "switch2" are present twice. It's valid as the relationship data is unique.

I would like to find a way to transform that dict into something like the following:

end_mapping = [{'switches': ['switch1', 'switch2'], 'relationships': [{'compute_name': 'compute001', 'servers': ['server1', 'server2']},
                                                                      {'compute_name': 'compute002', 'servers': ['server3', 'server4']}]},
               {'switches': ['switch3', 'switch4'], 'relationships': [{'compute_name': 'compute003', 'servers': ['server5', 'server6']},
                                                                      {'compute_name': 'compute004', 'servers': ['server7', 'server8', 'server9']}]},
               {'switches': ['switch5', 'switch6'], 'relationships': [{'compute_name': 'compute005', 'servers': ['server10', 'server11', 'server12']},
                                                                      {'compute_name': 'compute006', 'servers': ['server13', 'server14', 'server15', 'server16']}]}
              ]

In this data set, the switches key values are unique and I've "merged" the relationships dictionaries.

I've been banging my head around this but haven't found a way so far.

I have a starting point with:

new_list_of_dict = []
for mapping_dict_a in mappings:
    for mapping_dict_b in mappings:
        if set(mapping_dict_b['switches']) == set(mapping_dict_a['switches']):
            same = True
            mapping_dict_a['relationships'].append(mapping_dict_b['relationships'])
            new_list_of_dict.append(mapping_dict_a)
            break
        else:
            same = False
            print('NOT SAME!')

But it's not working so far (I'm sure it's wrong;) - and I end up with either an infinite loop or way too much data in the final_dict.

Any tips?

Thanks!

Answer 1

Hey so I came up with something that should work, though it is a hack. Let me know how it goes.

# given the mappings var you provided in your example:
switches = [item.get('switches') for item in mappings]
unique_switches = [list(x) for x in set(tuple(x) for x in switches)]

unique_mapping = []
for switch in unique_switches:
    relationship_list = []
    for item in mappings:
        if item.get('switches') == switch:
            relationship_list.append(item['relationships'])
    unique_mapping.append({'switches': switch, 'relationships': relationship_list})

unique_mapping should result in:

[{'switches': ['switch5', 'switch6'],
  'relationships': [[{'compute_name': 'compute005',
     'servers': ['server10', 'server11', 'server12']}],
   [{'compute_name': 'compute006',
     'servers': ['server13', 'server14', 'server15', 'server16']}]]},
 {'switches': ['switch1', 'switch2'],
  'relationships': [[{'compute_name': 'compute001',
     'servers': ['server1', 'server2']}],
   [{'compute_name': 'compute002', 'servers': ['server3', 'server4']}]]},
 {'switches': ['switch3', 'switch4'],
  'relationships': [[{'compute_name': 'compute003',
     'servers': ['server5', 'server6']}],
   [{'compute_name': 'compute004',
     'servers': ['server7', 'server8', 'server9']}]]}]

Answer 2

Here is a way usingfrozenset & why frozenset not set refer this post unhashable type: 'set'

results = {}

for v in mappings:
    key = frozenset(v['switches'])

    if results.get(key):
        results[key]['relationships'].append(v['relationships'])
    else:
        results[key] = v

print(list(results.values()))

[{'relationships': [{'compute_name': 'compute001',
                     'servers': ['server1', 'server2']},
                    [{'compute_name': 'compute002',
                      'servers': ['server3', 'server4']}]],
  'switches': ['switch1', 'switch2']},
...
}]