如果其中一个线程先结束，则结束python多线程

Question

So, I have the following code which im using to run the tasks in multiple functions at the same time:

        if __name__ == '__main__':
            po = Pool(processes = 10)
            resultslist = []
            i = 1
            while i <= 2:
                arg = [i]
                result = po.apply_async(getAllTimes, arg)
                resultslist.append(result)
                i += 1

            feedback = []
            for res in resultslist:
                multipresults = res.get()
                feedback.append(multipresults)

matchesBegin, matchesEnd = feedback[0][0], feedback[0][1]
TheTimes = feedback[1]

This works well for me. I'm currently using it to run two jobs at the same time.

But the problem is, i dont always need all the two simultaneously running jobs to complete before I move on to the next phases of the script. Sometimes, if the first job completes successfully and im able to confirm it by verifying whats in matchesBegin, matchesEnd, I want to be able to just move on and kill off the other job.

My issue is, i dont know how to do that.

Job 1 usually completes much faster than Job 2. So, what im trying to do here is, IF job 1 completes before Job 2, AND the content of the variables from Job 1 (matchesBegin, matchesEnd) is True, then, i want Job 2 to be blown away because I dont need it anymore. If i dont blow it away, it will only prolong the completion of the script. Job 2 should only be allowed to continue to run if results of the variables from Job 1 arent True.

Answer 1

I do not know all the details of your use case, but I would hope this provides you some direction. Essentially, what you've started with apply_async() could do that job, but you would also need to use its callback argument and evaluate incoming result to see if it fulfills your criteria and take a corresponding action if it does. I've hacked around your code a bit and got this:

class ParallelCall:
    def __init__(self, jobs=None, check_done=lambda res: None):
        self.pool = Pool(processes=jobs)
        self.pending_results = []
        self.return_results = []
        self.check_done = check_done

    def _callback(self, incoming_result):
        self.return_results.append(incoming_result)
        if self.check_done(incoming_result):
            self.pool.terminate()
        return incoming_result

    def run_fce(self, fce, *args, **kwargs):
        self.pending_results.append(self.pool.apply_async(fce,
                                                          *args, **kwargs,
                                                          callback=self._callback))

    def collect(self):
        self.pool.close()
        self.pool.join()
        return self.return_results

Which you could use like this:

def final_result(result_to_check):
    return result_to_check[0] == result_to_check[1]

if __name__ == '__main__':
    runner = ParallelCall(jobs=2, check_done=final_result)
    for i in range(1,3):
        arg = [i]
        runner.run_fce(getAllTimes, arg)

    feedback = runner.collect()

    TheTimes = feedback[-1]  # last completed getAllTimes call

What does it do? runner is an instance of ParallelCall (note: I've used only two workers as you seem to only run two jobs) which uses final_result() function to evaluate the result whether it is a suitable candidate for valid final result. In this case, it's first and second item are equal.

We use that to start getAllTimes two times like in your example above. It uses apply_async() just as you did, but we now also have a callback registered through which we pass the result when it becomes available. We also pass it through the function registered with check_done to see if we got an acceptable final result and if so (return value evaluates to True ) we just stop all the worker processes.

Disclaimer: this is not exactly what your example does, because the returning list is not in order in which function calls has taken place, but in which the results became available.

Then we collect() available results into feedback . This method closes the pool to not accept any further tasks ( close() ) and then waits for the workers to finish ( wait() ) (they could be stopped if one of the incoming results matched the registered criterion). Then we return all the results (either up to matching result or until all work has been done).

I've put this into ParallelCall class so that I can conveniently keep track of the pending and finished results as well know what my pool is. Default check_done is basically a (callable) nop .