Python - parallel commands

Question

How do I run several python commands in parallel in a python script ? As a simple example I have several sleep commands:

time.sleep(4)
time.sleep(6)
time.sleep(8)

I want all the above to be executed in parallel. I expect the control back when 8 seconds have passed (which is the max of all the sleeps above). The real commands will be different but want to get an idea with above.

In bash, I could have simple done:

sleep 4 &
pid1=$!
sleep 6 &
pid2=$!
sleep 8 &
pid3=$!
wait $pid1 $pid2 $pid3

Thanks.

Answer 1

One simple example, using multiprocessing:

import multiprocessing as mp
import time

pool = mp.Pool(3)
results = pool.map(time.sleep, [4, 6, 8] )

This spawns separate processes instead of creating separate threads in the same process as demonstrated in the answer by Steven Rumbalski. multiprocessing sidesteps the GIL (in cpython) meaning that you can execute python code simultaneously (which you can't do in cpython with threading). However, it has the downside that the information you send to the other processes must be pickleable, and sharing state between processes is a bit more complicated as well (although I could be wrong about the last point -- I've never really used threading ).

word of caution: Don't try this in the interactive interpreter

Answer 2

Take a look at the threading module. You could have something like:

import time,threading

def mySleep( sec ):
    time.sleep( sec )
t1 = threading.Thread( target=mySleep, args=(4,) )
t2 = threading.Thread( target=mySleep, args=(6,) )
t3 = threading.Thread( target=mySleep, args=(8,) )
t1.start()
t2.start()
t3.start()

# All threads running in parallel, now we wait
t1.join()
t2.join()
t3.join()

Answer 3

from threading import Thread

threads = [Thread(target=time.sleep, args=(secs,)) for secs in (4,6,8)]
for t in threads: t.start()
for t in threads: t.join()
print 'all threads done!'