How do you combine two similar nested dictionaries into one, each with some shared and unique nested elements (Python)?

Question

I have two large nested dictionaries that I need to merge into a single one:

dict1={
1: {"trait1: 32", "trait2": 43, "trait 3": 98},
2: {"trait1: 38", "trait2": 40, "trait 3": 95},
....
}

and

dict2={
1: {"trait1: 32", "trait2": 43, "trait 4": 54},
2: {"trait1: 38", "trait2": 40, "trait 4": 56},
....
}

and what I'd like to get is this:

dict3={
1: {"trait1: 32", "trait2": 43, "trait 3": 98, "trait 4": 54},
2: {"trait1: 38", "trait2": 40, "trait 3": 95, "trait 4": 56},
....
}

I've tried using:

dict3=dict(list(dict1.items()) + list(dict2.items()))

But it simply copies dict2 for me.

I've also tried looping through the "main" keys like this(I copied the first dictionary to become the final output):

dict3 = dict(dict1)

for key1 in dict3:
    for key2 in dict2:
        dict3[key1].update({"trait4": dict2[key2]["trait4"]})

But that doesn't work, only every few entries come out as expected in the output. And I'm fairly sure that my approach is wrong on this. Any help is appreciated!

Answer 1

To achieve your goal all you have to do is to check if dictionary contains key. You should define a function for example update_keys() which would take two arguments: dict1 and dict2 .

To check if dictionary has key simply write (as mentioned in this question ):

if key in dictionary:
    # Action you want to take if dictionary has key.

Therefore your solution would look like this ( note that deepcopy function is imported from copy module, as mentioned in Update 1 below ):

#!/usr/bin/env python3

from copy import deepcopy

def update_keys(dict1, dict2):
    result_dict = deepcopy(dict1)
    for key in dict2:
        if key in result_dict:
            for sub_key in dict2[key]:
                result_dict[key].update({sub_key: dict2[key][sub_key]})
        else: 
            result_dict.update({key: dict2[key]})
    return result_dict

dict3 = update_keys(dict1, dict2)

also to clarify things, you could iterate using values by using dictionary.items() like mentioned in this question , because in nested loop and multiple if statements you might get lost between all the variables.

#!/usr/bin/env python3

from copy import deepcopy

dict1={
1: {"trait1": 32, "trait2": 43, "trait3": 98},
2: {"trait1": 38, "trait2": 40, "trait3": 95}
}

dict2={
1: {"trait1": 32, "trait2": 43, "trait4": 54},
2: {"trait1": 38, "trait2": 40, "trait4": 56}
}

def update_keys(dict_one, dict_two):
    result_dict = deepcopy(dict_one)
    for key, value in dict_two.items():
        if key in result_dict:
            for sub_key, sub_value in value.items():
                if sub_key not in result_dict[key]:
                    result_dict[key].update({sub_key: sub_value})
        else:
            result_dict.update({key: value})
    return result_dict

dict3 = update_keys(dict1, dict2)

Update 1: Thanks to @shash678 I could improve my answer. Earlier passing dictionary to method and making copy by assigning new value created shallow copy as mentioned in this question topic . Therefore if dict1 is to be preserved, importing copy module and using deepcopy() function from it is necessary. Thanks to @shash678 , this answer does its job without modifying dict1.

Answer 2

def merge_dict(x,y):
    keys = list(set(x.keys() + y.keys()))
    result = {}
    for key in keys:
        if key in x.keys():
            z = x[key].copy()
            if key in y.keys():
                z.update(y[key])
            result[key] = z
        else:
            result[key] = y[key]
        result[k] = z
    return result


dict3 = merge_dict(dict1,dict2)

Answer 3

I would like to propose a merging that is very handy if the keys in both dicts can be assumed to be the same (for python 3.5+). If the keys are all the same, we may simply do this:

merged_dict = {}
for key in dict1.keys():
    merged_dict[key] = {**dict1[key], **dict2[key]}
# result: 
# merged_dict is {1: {'trait1': 32, 'trait2': 43, 'trait3': 98, 'trait4': 54},
#                 2: {'trait1': 38, 'trait2': 40, 'trait3': 95, 'trait4': 56}}

The double stars in front of a dict unpacks it, and since Python 3.5, this syntax will allow us to unpack dicts inside of a dict literal, effectively merging them. I am here assuming that in each value of the original dict is a dictionary itself. It's possible that you can do the unpacking of the nested dicts directly in some way (instead of using the for loop), but I don't know how (if anyone does, please comment!).

If we assume that the keys may differ, we gotta add some stuff, but there are no nested structures so I think it is still quite simple. Assume for example that dict2 has a key/value pair {3: {'trait5': 435, 'trait7': 42}}.

# find the shared keys, i.e. the intersection of the key sets
shared_keys = set(dict1.keys()).intersection(set(dict2.keys()))
merged_dict = {}
for key in shared_keys:
    merged_dict[key] = {**dict1[key], **dict2[key]}
# now add any key/value pairs only in dict1
for key in set(dict1.keys()) - shared_keys:
    merged_dict[key] = {**dict1[key]}
# and the same for dict2
for key in set(dict2.keys()) - shared_keys:
    merged_dict[key] = {**dict2[key]}
# result:
# merged_dict is {1: {'trait1': 32, 'trait2': 43, 'trait3': 98, 'trait4': 54}, 
#                 2: {'trait1': 38, 'trait2': 40, 'trait3': 95, 'trait4': 56}},
#                 3: {'trait5': 435, 'trait7': 42}}

I make use of set operations to get the shared keys and then the keys only in dict1 and dict2. For example, the intersection of {1,2,3} and {1,2} is {1,2}, and the set difference {1,2,3} - {1,2} = {3}.

I hope it's clear enough!