Python的collection.deque似乎在multiprocessing.Pool中不是线程安全的

Question

I am working in a small piece of code that is intended to work as follows:

• An arbitrary number of processes (eg 10) is to be run.
• Because I have two threads, I want to run them two at a time.
• I want them to read a value from a queue. The queue only contains two different values, and I want the two processes running at the same time to read these different values.

For instance, consider the next sample code:

import collections
import time

q = collections.deque()
q.append(0)
q.append(1)

def f(i):
    d = q.popleft()
    print d
    time.sleep(1)
    q.append(d)

from multiprocessing import Pool
t = Pool(2)
t.map(f, range(10))

When running this, I expect the first pair of processes to read 0 and 1 respectively. I don't really mind the order in which they are printed (either 0, 1 or 1, 0). However, the output is as follows:

0
0
1
1
0
0
1
1
0
1

Why is this happening if deque is thread-safe? Also, the first 8 values are printed 2 at a time every 1 second, but for the last 2 values there is a 1-second pause as well between them. To clarify this, let me draw 1-sec pauses as hyphens:

0
0
-
1
1
-
0
0
-
1
1
-
0
-
1

Why is this happening?

Thanks a lot.

Answer 1

Processes do not share memory.

In your example, each and every process works on its own copy of the deque. There is no synchronization between them at all.

You can use a multiprocessing.Queue object to achieve the correct result.