python2.7 threading pool is much faster than starting the thread one by one

Question

i did some testing on the thread pool and starting the thread one by one. It is much faster when using the thread pool. i can figure out why?

import threading
import Queue
import time

def stringFunction(value, out_queue):
    my_str = "This is string no. " + value
    time.sleep(5)
    out_queue.put(my_str)

my_queue = Queue.Queue()
i  = 1
threads = []
while i<10:
    thread = threading.Thread(stringFunction(str(i), my_queue))
    thread.start()
    threads.append(thread)
    i = i + 1

for thread in threads:
    thread.join()


func_value = list(my_queue.queue)
print func_value

[Running] python -u "c:\Users\eguozqu\git\test\thread_test.py" ['This is string no. 1', 'This is string no. 2', 'This is string no. 3', 'This is string no. 4', 'This is string no. 5', 'This is string no. 6', 'This is string no. 7', 'This is string no. 8', 'This is string no. 9']

[Done] exited with code=0 in 45.353 seconds

from multiprocessing.pool import ThreadPool as Pool
import time
class someClass(object):
   def __init__(self):
       pass
   def f(self, x):
       time.sleep(5)
       return x*x

   def go(self):
      p = Pool(10)
      sc = p.map(self, range(10))
      print sc

   def __call__(self, x):   
     return self.f(x)

sc = someClass()
sc.go()

[Running] python -u "c:\Users\eguozqu\git\test\process_test.py" [0, 1, 4, 9, 16, 25, 36, 49, 64, 81]

[Done] exited with code=0 in 5.393 seconds

Answer 1

When you do threading.Thread(stringFunction(str(i), my_queue)) you are calling stringFunction and passing the result to threading.Thread 's initialiser instead of passing the function itself.

This means that each time you create a thread you first wait for that function call to complete.

What you probably want is threading.Thread(target=stringFunction, args=(str(i), my_queue)) .