Simultaneously call functions with different parametrs in Python

Question

I found a lot of examples, but not mine. Please help me here:

I have a functions that requires a parameter (an URL) and I need to call this function several times (with different parameters) at the same time to get something like that:

 function started with param: rsync://.... (1)
 function started with param: rsync://.... (2)
 function started with param: rsync://.... (3)
 ....
 function (k) with param rsync://....has finished
 .....
 function started with param: rsync://.... (n)
 function (1) with param rsync://....has finished
 ....
 function (n) with param rsync://....has finished

Those functions are independent, they don't interact with any common objects or something like that (excluding imports). I need to run them at the same time (n of those functions). It doesn't matter the order how they would finish.

Answer 1

Use Threads

For instance, you can use:

import threading
my_thread = threading.Thread(target=your_function, args=(), kwargs={})  # This can go inside a loop, providing different args each time.
my_thread.start()

Visit the link at top of the answer for more info.

Answer 2

If you want to stay free concerning the contents of this functions, use threads as Raydel mentions, or use multiprocessing if the functions are supposed to be CPU bound rather than IO bound.

If your task is just to call subprocesses, you can

• start the subprocess with sp = subprocess.Popen()
• keep this sp , kind of a handle to it
• then they run in parallel
• poll() them until they are done
• then you know when they have finished.