Stop a process when error occur in multiprocessing

Question

I have created a 3 process in python. I have attached a code. Now I want to stop the execution of running p2,p3 process because I got an error due to p1 process.I have idea to add p2.terminate(),I don't know where to add in this case. Thanks in advance.

def table(a):
     try:
        for i in range(100):
            print(i,'x',a,'=',a*i)
     except:
        print("error")

processes = []
p1= multiprocessing.Process(target = table,args=['s'])
p2= multiprocessing.Process(target = table,args=[5])
p3= multiprocessing.Process(target = table,args=[2])
p1.start()
p2.start()
p3.start()

processes.append(p1)
processes.append(p2)
processes.append(p3)

for process in processes:
    process.join()```

Answer 1

to stop a all cores when one core has faced with error, i use this code block:

processes = []
for j in range(0, n_core):
    p = multiprocessing.Process(target=table, args=('some input',))
    processes.append(p)
    time.sleep(0.1)
    p.start()

flag = True
while flag:
    flag = False
    for p in processes:
        if p.exitcode == 1:
            for z in processes:
                z.kill()
            sys.exit(1)

        elif p.is_alive():
            flag = True

for p in processes:
    p.join()

Answer 2

To stop any given process once one of the process terminates due to an error, first set up your target table() to exit with an appropriate exitcode > 0

def table(args):
    try:
        for i in range(100):
            print(i,'x', a ,'=', a*i)
     except:
        sys.exit(1)
    sys.exit(0)

Then you can start your processes and poll the processes to see if any one has terminated.

#!/usr/bin/env python3
# coding: utf-8

import multiprocessing
import time
import logging
import sys

logging.basicConfig(level=logging.INFO, format='[%(asctime)-15s] [%(processName)-10s] %(message)s', datefmt='%Y-%m-%d %H:%M:%S')


def table(args):
    try:
        for i in range(5):
            logging.info('{} x {} = {}'.format(i, args, i*args))
            if isinstance(args, str):
                raise ValueError()
            time.sleep(5)
    except:
        logging.error('Done in Error Path: {}'.format(args))
        sys.exit(1)
    logging.info('Done in Success Path: {}'.format(args))
    sys.exit(0)


if __name__ == '__main__':
    p1 = multiprocessing.Process(target=table, args=('s',))
    p2 = multiprocessing.Process(target=table, args=(5,))
    p3 = multiprocessing.Process(target=table, args=(2,))
    processes = [p1, p2, p3]
    for process in processes:
        process.start()

    while True:
        failed = []
        completed = []
        for process in processes:
            if process.exitcode is not None and process.exitcode != 0:
                failed.append(process)
        if failed:
            for process in processes:
                if process not in failed:
                    logging.info('Terminating Process: {}'.format(process))
                    process.terminate()
            break
        if len(completed) == len(processes):
            break
        time.sleep(1)

Essentially, you are using terminate() to stop the remaining processes that are still running.

Answer 3

First, I have modified function table to throw an exception that is not caught when the argument passed to it is 's' and to delay.1 seconds otherwise before printing to give the main process a chance to realize that the sub-process through an exception and can cancel the other processes before they have started printing. Otherwise, the other processes will have completed before you can cancel them. Here I am using a process pool, which supports a terminate method that conveniently terminates all submitted, uncompleted tasks without having to cancel each one individually (although that is also an option).

The code creates a multiprocessing pool of size 3 since that is the number of "tasks" being submitted and then uses method apply_async to submit the 3 tasks to run in parallel (assuming you have at least 3 processors). apply_sync returns an AsyncResult instance whose get method can be called to wait for the completion of the submitted task and to get the return value from the worker function table , which is None for the second and third tasks submitted and of no interest, or will throw an exception if the worker function had an uncaught exception, which is the case with the first task submitted:

import multiprocessing
import time

def table(a):
    if a == 's':
        raise Exception('I am "s"')
    time.sleep(.1)
    for i in range(100):
        print(i,'x',a,'=',a*i)

# required for Windows:
if __name__ == '__main__':
    pool = multiprocessing.Pool(3) # create a pool of 3 processes
    result1 = pool.apply_async(table, args=('s',))
    result2 = pool.apply_async(table, args=(5,))
    result3 = pool.apply_async(table, args=(2,))
    try:
        result1.get() # wait for completion of first task
    except Exception as e:
        print(e)
        pool.terminate() # kill all processes in the pool
    else:
        # wait for all submitted tasks to complete:
        pool.close()
        pool.join()
        """
        # or alternatively:
        result2.get() # wait for second task to finish
        result3.get() # wait for third task to finish
        """

Prints:

I am "s"