Remove duplicates from a nested dictionary

Question

I have a dictionary as follows:

   data =` {0: {key1: {key2 : [value1 , value2 ,value3]}}}`

Now I have to remove the duplicate value from the list, which is inside the nested dictionary. Also, the function should be recursive and should pass two arguments, like :

remove_duplicates(dict , 'key2')


def remove_duplicate(dict, keys):
    for key, value in duplicate_values_dict.items():
        if key in dict_keys:
            duplicate_values_dict[key] = set(value)
     return duplicate_values_dict

Here, I am unable to pass tuple of key, as in the question.

The function should remove the duplicate value from the specified key and append it in a new dictionary.

How do I solve the problem?

Answer 1

The data you have shared contains nested dictionary, So I make a recursive function:

def remove_duplicate(duplicate_values_dict, dict_keys):
    if dict_keys in duplicate_values_dict.keys():
        if isinstance(duplicate_values_dict[dict_keys], list):
            duplicate_values_dict[dict_keys] = list(set(duplicate_values_dict[dict_keys]))
    for key, value in duplicate_values_dict.items():
        if isinstance(value, dict):
            remove_duplicate(value,dict_keys)

you can call this function using the dictionary and a key:

remove_duplicate(data,'last_name')

the key 'last_name' contains duplicate : ['GUYATT', 'GUYATT'] initially. After calling remove_duplicate(data,'last_name') you will find your 'data' contains only ['GUYATT'] for the key 'last_name'

print(data)

output:

{0: {key1 : {key2 :[value1,value2]}}}

from output you will find that the key 'last_name' contains['GUYATT']