python migrate data from one dict into another dict using a template

Question

I want to migrate the data from a dict into another dict for which I have a template.

Original dict:

persons = {
    'names': ['Greg', 'Milla', 'Paul', 'Linda'],
    'ages': [43, 24, 55, 14],
}

Template dict:

persons_template = {
    'data': {
        'name': '',
        'age': '',
    }
}

Migration script:

persons_new = {}
for idx, name in enumerate(persons['names']):
    person = persons_template
    person['data']['name'] = persons['names'][idx]
    person['data']['age'] = persons['ages'][idx]
    print(person)
    persons_new[idx] = person
persons_new

The executed code results as follows:

{'data': {'name': 'Greg', 'age': 43}}
{'data': {'name': 'Milla', 'age': 24}}
{'data': {'name': 'Paul', 'age': 55}}
{'data': {'name': 'Linda', 'age': 14}}

{0: {'data': {'name': 'Linda', 'age': 14}},
 1: {'data': {'name': 'Linda', 'age': 14}},
 2: {'data': {'name': 'Linda', 'age': 14}},
 3: {'data': {'name': 'Linda', 'age': 14}}}

The first output from the print statement shows correctly the data sets. The second output shows the new dict "persons_new". I can't figure out why "persons_new" only has the last data set from the original dict?

Thanks and regards, carlo

Answer 1

person = person_template doesn't do what you think it does. You're binding person to the same underlying dictionary that person_template is bound to. You're not making a copy or anything. Any changes you apply to person will also be applied to person_template , and vice-versa.

Alternatively, the dict built-in accepts a collection of tuples, which will be unpacked into key-value pairs. If you write a generator that yields key-value tuples, your dictionary can be generated in the following way:

def generate_dict():
    for index, (name, age) in enumerate(zip(persons["names"], persons["ages"])):
        yield index, {
            "data": {
                "name": name,
                "age": age
            }
        }

persons_new = dict(generate_dict())
print(persons_new)

Output:

{0: {'data': {'name': 'Greg', 'age': 43}}, 1: {'data': {'name': 'Milla', 'age': 24}}, 2: {'data': {'name': 'Paul', 'age': 55}}, 3: {'data': {'name': 'Linda', 'age': 14}}}
>>> 

Answer 2

That is due to the named reference. Try printing persons_template as well at each iteration, you'll know what's going wrong. Basically, what happens is, when you use assignment = operators for sequence, Python creates a named reference instead of creating a separate copy, in your case, person is also making reference to the same memory location as persons_template . You can verify this printing person is persons_template inside the loop. To illustrate it, you can try something like:

x = (2,3,)
y = x
x is y
Out[127]: True

Or

x = {'a':1}
y = x
y is x
Out[128]: True

To avoid that, you can copy the dictionary to another variable using copy.deepcopy

Modified version:

import copy
persons_new = {}
for idx, name in enumerate(persons['names']):
    person = copy.deepcopy(persons_template)
    person['data']['name'] = persons['names'][idx]
    person['data']['age'] = persons['ages'][idx]
    print(person)
    persons_new[idx] = copy.deepcopy(person)
persons_new

OUTPUT

{'data': {'name': 'Greg', 'age': 43}}
{'data': {'name': 'Milla', 'age': 24}}
{'data': {'name': 'Paul', 'age': 55}}
{'data': {'name': 'Linda', 'age': 14}}
Out[124]: 
{0: {'data': {'name': 'Greg', 'age': 43}},
 1: {'data': {'name': 'Milla', 'age': 24}},
 2: {'data': {'name': 'Paul', 'age': 55}},
 3: {'data': {'name': 'Linda', 'age': 14}}}

Answer 3

Since you are referencing person every time you iterate, the value of person updates itself every time, and you end up with 4 references in your dictionary to the final version of person, ie Linda.

If you want to copy the values from the dictionary, you can use the following:

from copy import deepcopy

persons_new = {}

for idx, name in enumerate(persons['names']):
    person = persons_template
    person['data']['name'] = persons['names'][idx]
    person['data']['age'] = persons['ages'][idx]
    print(person)
    persons_new[idx] = deepcopy(person)
persons_new

This should give you the result you expect:

{'data': {'name': 'Greg', 'age': 43}}
{'data': {'name': 'Milla', 'age': 24}}
{'data': {'name': 'Paul', 'age': 55}}
{'data': {'name': 'Linda', 'age': 14}}
{0: {'data': {'name': 'Greg', 'age': 43}},
 1: {'data': {'name': 'Milla', 'age': 24}},
 2: {'data': {'name': 'Paul', 'age': 55}},
 3: {'data': {'name': 'Linda', 'age': 14}}}