Python Multiprocessing Start Process in Module Other Than Main

Question

I have three modules, worker , master , and MainTests . I'm running the MainTests module as the main script. In MainTests , I call master.run() , inside of which I need to spawn multiple worker processes. Is this possible? In all the python multiprocessing tutorials I have come across, processes are started in the main module. If this is possible, could someone provide an example as to what this might look like?

This is what I have attempted so far:

Worker.py

import time

class Worker(object):
    def __init__(self):
        super(Worker, self).__init__()
    def run(self):
        time.sleep(5)
        print("worker done with run")
        return

Master.py:

import multiprocessing

class Master(object):
    def __init__(self, workers_array):
        super(Master, self).__init__()
        self.workers_array = workers_array
    def run(self):
        process_arr = [multiprocessing.Process(worker.run()) for worker in self.workers_array]
        [worker_process.start() for worker_process in process_arr]

MainTests.py

from Worker import *
from Master import *

workers_array = [Worker() for i in range(5)]
master = Master(workers_array)
master.run()

Two issues arise:

1. Workers seem to be running sequentially, one by one executing run() rather than running in parallel.
2. Workers seem to keep repeating runs. I would expect that after the workers complete their runs, the program ends, but it keeps going.

Thanks in advance for any help.

Answer 1

I'm not sure if you're still locking for an answer but you just have to put the "entry point" of your main program in this if statement:

if __name__ == "__main__":
    main()

This way you can start a process in an imported module. For further information check out this bit of the docs and obviously everything else in there :)

In your example it would be:

from Worker import *
from Master import *

if __name__ == __main__:
    workers_array = [Worker() for i in range(5)]
    master = Master(workers_array)
    master.run()

This has worked for me. Hope I could help.