无法正确修改 Python class 中的字典

Question

I've incurred in a strange problem while I was trying to modify a list of dictionaries in a class. This is the smallest reproducible code that shows the behaviour:

from itertools import product

class Test():
    def __init__(self, grid):
        self.grid = grid
        self.pc =  [dict(zip(self.grid, v)) for v in product(*self.grid.values())]
        for i in range(0, len(self.pc)):
            self.pc[i]['sim_options']['id'] = i

grid = {
    'k': [5, 10, 15, 20],
    'sim_options': [
        {'name': 'cosine', 'batched': True},
        {'name': 'pearson', 'batched': True}
    ]
}
t = Test(grid)

What I'd expect as output is the following:

[{'k': 5, 'sim_options': {'name': 'cosine', 'batched': True, 'id': 0}},
 {'k': 5, 'sim_options': {'name': 'pearson', 'batched': True, 'id': 1}},
 {'k': 10, 'sim_options': {'name': 'cosine', 'batched': True, 'id': 2}},
 {'k': 10, 'sim_options': {'name': 'pearson', 'batched': True, 'id': 3}},
 {'k': 15, 'sim_options': {'name': 'cosine', 'batched': True, 'id': 4}},
 {'k': 15, 'sim_options': {'name': 'pearson', 'batched': True, 'id': 5}},
 {'k': 20, 'sim_options': {'name': 'cosine', 'batched': True, 'id': 6}},
 {'k': 20, 'sim_options': {'name': 'pearson', 'batched': True, 'id': 7}}]

and yet I get:

[{'k': 5, 'sim_options': {'name': 'cosine', 'batched': True, 'id': 6}},
 {'k': 5, 'sim_options': {'name': 'pearson', 'batched': True, 'id': 7}},
 {'k': 10, 'sim_options': {'name': 'cosine', 'batched': True, 'id': 6}},
 {'k': 10, 'sim_options': {'name': 'pearson', 'batched': True, 'id': 7}},
 {'k': 15, 'sim_options': {'name': 'cosine', 'batched': True, 'id': 6}},
 {'k': 15, 'sim_options': {'name': 'pearson', 'batched': True, 'id': 7}},
 {'k': 20, 'sim_options': {'name': 'cosine', 'batched': True, 'id': 6}},
 {'k': 20, 'sim_options': {'name': 'pearson', 'batched': True, 'id': 7}}]

I don't get what I'm doing wrong, am I not iterating over a list, accessing the 'sim_options' field of the i-th list and creating a new key-value pair ('id':i) in that dictionary?

Answer 1

Seems like the dictionaries are getting updated by reference, so whatever you do to one, happens to the other.

You can check this with the id() function:

for i in range(0, len(self.pc)):
    print(id(self.pc[i]['sim_options']))
    self.pc[i]['sim_options']['id'] = i

Which gives me the same repeating reference identities( 2078935351104 and 2078964975104 ) for both dictiionaries in 'sim_options' :

2078935351104
2078964975104
2078935351104
2078964975104
2078935351104
2078964975104
2078935351104
2078964975104

One way to get around this is to copy options, which will give you a different reference identity. I've modified your code slightly to make this possible using copy() . It also uses enumerate() to loop over the indices and items, which is much nicer to use than range(len(...)) when you need both.

from itertools import product
from pprint import pprint

class Test:
    def __init__(self, grid):
        self.grid = grid

        self.pc = []
        for i, (k, options) in enumerate(product(self.grid["k"], self.grid["sim_options"])):
            temp = {"k": k, "sim_options": options.copy()}
            temp["sim_options"]["id"] = i
            self.pc.append(temp)

    def get(self):
        return self.pc

grid = {
    "k": [5, 10, 15, 20],
    "sim_options": [
        {"name": "cosine", "batched": True},
        {"name": "pearson", "batched": True},
    ],
}

t = Test(grid)

pprint(t.get())

Or just build a new list of dictionaries with a list comprehension. I usually find this way much more preferable.

self.pc = [
    {"k": k, "sim_options": {**option, "id": idx}}
    for idx, (k, option) in enumerate(
        product(self.grid["k"], self.grid["sim_options"])
    )
]

Output:

[{'k': 5, 'sim_options': {'batched': True, 'id': 0, 'name': 'cosine'}},
 {'k': 5, 'sim_options': {'batched': True, 'id': 1, 'name': 'pearson'}},
 {'k': 10, 'sim_options': {'batched': True, 'id': 2, 'name': 'cosine'}},
 {'k': 10, 'sim_options': {'batched': True, 'id': 3, 'name': 'pearson'}},
 {'k': 15, 'sim_options': {'batched': True, 'id': 4, 'name': 'cosine'}},
 {'k': 15, 'sim_options': {'batched': True, 'id': 5, 'name': 'pearson'}},
 {'k': 20, 'sim_options': {'batched': True, 'id': 6, 'name': 'cosine'}},
 {'k': 20, 'sim_options': {'batched': True, 'id': 7, 'name': 'pearson'}}]