Python multiprocessing.Process 自行调用 join

Question

I have this code:

class ExtendedProcess(multiprocessing.Process):
    def __init__(self):
        super(ExtendedProcess, self).__init__()
        self.stop_request = multiprocessing.Event()

    def join(self, timeout=None):
        logging.debug("stop request received")
        self.stop_request.set()
        super(ExtendedProcess, self).join(timeout)

    def run(self):
        logging.debug("process has started")
        while not self.stop_request.is_set():
            print "doing something"
        logging.debug("proc is stopping")

When I call start() on the process it should be running forever, since self.stop_request() is not set. After some miliseconds join() is being called by itself and breaking run. What is going on!? why is join being called by itself?

Moreover, when I start a debugger and go line by line it's suddenly working fine.... What am I missing?

OK, thanks to ely's answer the reason hit me:

There is a race condition -

1. new process created...
2. as it's starting itself and about to run logging.debug("process has started") the main function hits end.
3. main function calls sys exit and on sys exit python calls for all finished processes to close with join().
4. since the process didn't actually hit "while not self.stop_request.is_set()" join is called and "self.stop_request.set()". Now stop_request.is_set and the code closes.

Answer 1

As mentioned in the updated question, this is because of a race condition. Below I put an initial example highlighting a simplistic race condition where the race is against the overall program exit, but this could also be caused by other types of scope exits or other general race conditions involving your process.

I copied your class definition and added some "main" code to run it, here's my full listing:

import logging
import multiprocessing
import time


class ExtendedProcess(multiprocessing.Process):
    def __init__(self):
        super(ExtendedProcess, self).__init__()
        self.stop_request = multiprocessing.Event()

    def join(self, timeout=None):
        logging.debug("stop request received")
        self.stop_request.set()
        super(ExtendedProcess, self).join(timeout)

    def run(self):
        logging.debug("process has started")
        while not self.stop_request.is_set():
            print("doing something")
            time.sleep(1)
        logging.debug("proc is stopping")


if __name__ == "__main__":
    p = ExtendedProcess()
    p.start()
    while True:
        pass

The above code listing runs as expected for me using both Python 2.7.11 and 3.6.4. It loops infinitely and the process never terminates:

ely@eschaton:~/programming$ python extended_process.py 
doing something
doing something
doing something
doing something
doing something
... and so on

However, if I instead use this code in my main section, it exits right away (as expected):

if __name__ == "__main__":
    p = ExtendedProcess()
    p.start()

This exits because the interpreter reaches the end of the program, which in turn triggers automatically destroying the p object as it goes out of scope of the whole program.

Note this could also explain why it works for you in the debugger. That is an interactive programming session, so after you start p , the debugger environment allows you to wait around and inspect it ... it would not be automatically destroyed unless you somehow invoked it within some scope that is exited while stepping through the debugger.

Just to verify the join behavior too, I also tried with this main block:

if __name__ == "__main__":
    log = logging.getLogger()
    log.setLevel(logging.DEBUG)
    p = ExtendedProcess()
    p.start()
    st_time = time.time()
    while time.time() - st_time < 5:
        pass
    p.join()
    print("Finished!")

and it works as expected:

ely@eschaton:~/programming$ python extended_process.py 
DEBUG:root:process has started
doing something
doing something
doing something
doing something
doing something
DEBUG:root:stop request received
DEBUG:root:proc is stopping
Finished!