Using a for loop along with multi threading python

Question

I have a parent script that runs 10 child scripts. These scripts are identical apart from the 'name', each script calls a different api, saves to a different folder etc that depends on name.

I originally just copied the script 10 times however I now realise this is ridiculous as if I make one upgrade I have to do it to all 10.

I'm quite new to threading and am not sure if this is allowed, but can I set up a for loop or similar that calls the script and places the 'name' inside:

Ie

#parent.py
from threading import Thread
import sys
names =  ['BTC', 'BTS', 'ETH', 'CLAM', 'DOGE', 'FCT', 'MAID', 'STR', 'XMR', 'XRP' ]

for name in names: 
    sys.path.append('/python/loanrates/'+name)
    import name
    Thread(target=name.main(name)).start()
    Thread(target=name.main(name)).join()

Answer 1

To begin with, you should consider using the multiprocessing module , as threading is misleading in Python (in some sense, Python is currently single threaded).

Using this module, it's very easy to do "push the loop" on to the module.

First, define a pool:

# start 4 worker processes
pool = Pool(processes=4)         

Now, if you have a function f that runs on a name, you can use

pool.map(f, names)     

This will return only when f was applied to each item in names , using the Pool of 4 processes (in this case). Ie, it's like a "magical" parallel for loop

"parallel for" name in names:
    f(name)

Answer 2

You can have list of threads , start them in a loop and then wait for them to join in another for loop:

from threading import Thread
import sys
names =  ['BTC', 'BTS', 'ETH', 'CLAM', 'DOGE', 'FCT', 'MAID', 'STR', 'XMR', 'XRP' ]
threads = []
for name in names: 
    sys.path.append('/python/loanrates/'+name)
    import name
    T = Thread(target=name.main(name))
    threads.append(T)

for thread_ in threads:
    thread_.start()

for thread_in threads:
    thread_.join()