I'm wondering if dict.update() is python thread safe. I've read the related questions, but none of them exactly addresses my question.
My question is very specific and simple. For example, I already have a local dictionary d2
. I simply need to update the global dictionary d
with d2
as shown below. d
starts out empty and fills up with different threads. The d2
in each thread may have overlapping entries with d
(don't think this matters). Is it thread safe?
import dis
def f(d):
d2 = {1:2, 3:4}
d.update(d2)
print(dis.dis(f))
The bytecode looks like the following:
10 0 LOAD_CONST 1 (2)
2 LOAD_CONST 2 (4)
4 LOAD_CONST 3 ((1, 3))
6 BUILD_CONST_KEY_MAP 2
8 STORE_FAST 1 (d2)
11 10 LOAD_FAST 0 (d)
12 LOAD_ATTR 0 (update)
14 LOAD_FAST 1 (d2)
16 CALL_FUNCTION 1
18 POP_TOP
20 LOAD_CONST 0 (None)
22 RETURN_VALUE
It looks like 16 CALL_FUNCTION
is the atomic function that updates the dictionary. So it should be thread safe?
If the keys are compositions of builtin hashable types, generally "yes", .update()
is thread-safe. In particular, for your example with integers keys, yes.
But in general , no. Looking up a key in a dict can invoke arbitrary user-defined Python code in user-supplied __hash__()
and __eq__()
methods, and those can do anything at all - including performing their own mutations on the dicts involved. As soon as the implementation invokes Python code, other threads can run too, including threads that may be mutating d1
and/or d2
too.
That's not a potential problem for the builtin hashable types (ints, strings, floats, tuples, ...) - their implementations to compute hash codes and decide equality are purely functional (deterministic and no side effects) and don't release the GIL (global interpreter lock).
That's all about CPython (the C implementation of Python). The answer may differ under other implementations! The Language Reference Manual is silent about this.
You could look into locked-dict if you're ok with using an external library.
From their readme:
Dict to allow context managed thread safe and mutable iterations through a lock.
For example from their tests :
pip install locked-dict
import locked_dict
expected = 0
d = locked_dict.LockedDict()
assert len(d) == expected
assert bool(d) is False
assert d is not True
assert hasattr(d, '_lock')
empty_d = {}
assert d == empty_d
plain_old_d = {999: 'plain old dict', 12345: 54321}
assert d != plain_old_d
with d as m:
assert len(m) == expected
assert bool(m) is False
assert m is not True
assert hasattr(m, '_lock')
assert m != plain_old_d
assert m == empty_d
m[0] = ['foo']
expected += 1
assert len(m) == expected
assert bool(m) is True
assert m is not False
assert m != plain_old_d
assert m != empty_d
m.clear()
expected -= 1
assert len(m) == expected
assert bool(m) is False
assert m is not True
assert m != plain_old_d
assert m == empty_d
Take not this library is 3 years old, although it may still be relevent to your use case
The technical post webpages of this site follow the CC BY-SA 4.0 protocol. If you need to reprint, please indicate the site URL or the original address.Any question please contact:yoyou2525@163.com.