捕获python子进程异常

Question

I have a python script where i am calling another script using subprocess as below.

        sp = subprocess.Popen("script.py --arg1 --arg2', cwd=GIVEN_PATH, shell=True, stdout=subprocess.PIPE,  stderr=subprocess.STDOUT)

        while sp.poll() is None:
            for out in iter(sp.stdout.readline,''):
                self.log(out.rstrip())

This is working fine for me but i want to get any exception from the script.py. I know we can get the retcode but i actually need to get the full exception information.

If script.py raises

         raise IOERROR("got an exception")

then i need to know this information. simailar way we get the sys.exc_info() etc.

is there a way i can do this?

Thanks in Advance

Answer 1

No, you can't get the sys.exc_info() from a subprocess -- by the time you're able to see the return code, objects in memory (including stack traces) of the child process are already gone.

What you can do is parse the text your subprocess writes to stderr, by providing a separate pipe for the stderr output and reading that. Assuming that your program never writes to stderr otherwise, the only text that will show up in that stream will be the error message and stack trace.

You'll want to be cautious with this approach, though, because if the process writes more text to stderr than you can buffer, you'll deadlock. The best solution is to have a separate thread to read from both stdout and stderr in parallel.

Answer 2

If you want to get sys.exc_info() then you could import the module and run functions from it instead of using subprocess module. You could use multiprocessing module if you need to run in a different process.

Another possibility is to use execfile() to run the module as a script in the same process as the parent script. If you want to pass command-line arguments to the script then you could manipulate sys.argv global list before calling execfile() .

Or you could combine subprocess + execfile methods: write a wrapper script that uses execfile to run the child script and serializes all exceptions (eg, using pickle ) and run the wrapper script using subprocess in your parent script. Capture stderr separately, to be able to unpickle child's exception info.