[英]Multi-threaded Python application slower than single-threaded implementation
[英]Python's multi-threaded code is slower than single-threaded
我首先在生产代码中观察到此问题,然后制作了一个原型:
import threading, Queue, time, sys
def heavyfunc():
''' The idea is just to load CPU '''
sm = 0
for i in range(5000):
for j in range(5000):
if i + j % 2 == 0:
sm += i - j
print "sm = %d" % sm
def worker(queue):
''' worker thread '''
while True:
elem = queue.get()
if elem == None: break
heavyfunc() # whatever the elem is
starttime = time.time()
q = Queue.Queue() # queue with tasks
number_of_threads = 1
# create & start number_of_threads working threads
threads = [threading.Thread(target=worker, args=[q]) for thread_idx in range(number_of_threads)]
for t in threads: t.start()
# add 2 working items: they are estimated to be computed in parallel
for x in range(2):
q.put(1)
for t in threads: q.put(None) # Add 2 'None' => each worker will exit when gets them
for t in threads: t.join() # Wait for every worker
#heavyfunc()
elapsed = time.time() - starttime
print >> sys.stderr, elapsed
heavyfunc()的想法只是加载CPU,而没有任何同步和依赖关系。
使用1个线程时,平均需要花费4.14秒。使用2个线程时,平均需要花费6.40秒。不使用任何线程时,计算heavyfunc()的平均需要花费2.07秒(多次测量,精确地为4.14 / 2,因为如果有1个线程和2个任务)。
如果有2个线程,我预计2个具有heavyfunc()的工作将花费2.07秒。 (我的CPU是i7 =>有足够的内核)。
这是CPU监视器的屏幕截图,也给出了没有真正的多线程的想法:
我的思想错误在哪里? 如何创建不干扰的n个线程?
CPython不会一次在多个内核上执行字节码。 cpu绑定的多线程代码毫无意义。 全局解释器锁(GIL)可以保护进程中的所有引用计数,因此一次只能有一个线程使用Python对象。
您看到的性能较差,因为一次只能运行一个线程,但是现在您还在更改线程上下文。
声明:本站的技术帖子网页,遵循CC BY-SA 4.0协议,如果您需要转载,请注明本站网址或者原文地址。任何问题请咨询:yoyou2525@163.com.