[英]Python: Why different threads get their own series of values from one generator?
I'm learning multithreading in Python. 我正在学习Python中的多线程。 I want to know how to provide data to multiple threads using generators.
我想知道如何使用生成器向多个线程提供数据。 Here's what I wrote:
这是我写的:
import threading
data = [i for i in xrange(100)]
def generator():
for i in data:
yield i
class CountThread(threading.Thread):
def __init__(self, name):
threading.Thread.__init__(self)
self.name = name
def run(self):
for i in generator():
print '{0} {1}'.format(self.name, i)
a = CountThread('a')
b = CountThread('b')
a.start()
b.start()
I thought the list would only be iterated for once. 我以为列表只会被迭代一次。 But it seems that each thread is interating through the list independently.
但似乎每个线程都是独立地通过列表进行交互。
output: 输出:
a 0
a 1
a 2
b 0
a 3
a 4
b 1
b 2
a 5
a 6
a 7
a 8
a 9
b 3
b 4
b 5
b 6
b 7
b 8
...
(200 lines)
What is the underlying reason for this? 这是什么原因? How can I rewrite the program so that the values in the list will only be printed once.
如何重写程序,以便列表中的值只打印一次。
You instantiate a new generator in each thread in run
function with this: 您
run
使用以下命令在run
function中的每个线程中实例化一个新生成器:
for i in generator():
each generator
call returns a new instance of generator: 每个
generator
调用返回一个新的生成器实例:
>>> data = [i for i in xrange(10)]
>>> a, b = generator(), generator()
>>> id(a), id(b)
(37528032, 37527952)
Here a
and b
have different ids, and generate identical results even without threading: 这里
a
和b
具有不同的id,即使没有线程也会生成相同的结果:
>>> list(a), list(b)
([0, 1, 2, 3, 4, 5, 6, 7, 8, 9], [0, 1, 2, 3, 4, 5, 6, 7, 8, 9])
Note however, that generators aren't thread safe, it is tricky to use them in threaded application. 但请注意,生成器不是线程安全的,在线程应用程序中使用它们很棘手。 You have to worry about locking, see example , or you'll get
ValueError: generator already executing
error once in a while. 您必须担心锁定, 请参阅示例 ,否则您将获得
ValueError: generator already executing
偶尔会ValueError: generator already executing
错误。 Alternatively you can use Queue.Queue for thread communication. 或者,您可以使用Queue.Queue进行线程通信。
Can you just use Python Queue class. 你能使用Python Queue类吗? I believe that is thread safe.
我相信这是线程安全的。 Basically you could just use your generator to fill the Queue and then have each thread pull from it.
基本上你可以使用你的生成器来填充队列,然后让每个线程从中拉出来。
#!/usr/bin/python
import Queue
import threading
queueOfNumbers = Queue.Queue()
data = [i for i in xrange(100)]
def generator():
for i in data:
yield i
class CountThread(threading.Thread):
def __init__(self, name, queue):
threading.Thread.__init__(self)
self.name = name
self.queue = queue
def run(self):
i = self.queue.get()
print '%s %s' % (self.name, i)
a = CountThread('a', queueOfNumbers)
b = CountThread('b', queueOfNumbers)
a.start()
b.start()
for num in generator():
queueOfNumbers.put(num)
queueOfNumbers.join()
http://www.ibm.com/developerworks/aix/library/au-threadingpython/ http://www.ibm.com/developerworks/aix/library/au-threadingpython/
声明:本站的技术帖子网页,遵循CC BY-SA 4.0协议,如果您需要转载,请注明本站网址或者原文地址。任何问题请咨询:yoyou2525@163.com.