How can I retrospectively debug a python exception

Question

I'm looking for a way to debug a python exception "retrospectively". Essentially if my program raises an exception that isn't handled, I want it to save off the program state so I can go back later and debug the problem.

I've taken a look at the pdb docs, and it seems that you can do this, but only if you can interact with the program at the point of the exception. This won't work for me as the program will be run in the background (without a controlling terminal).

My first (doomed!) approach was to put a try/except block at the highest level of my program, and in the except block extract the traceback object from the current exception and write it to disk using pickle. I planned to then write a separate program that would unpickle the object and use pdb.post_mortem to debug the crashed program. But traceback objects aren't pickleable, but I wouldn't expect that to work anyway, as it wouldn't save off the entire program state.

Answer 1

As far as I know, there isn't any way to do what you're asking. That said, it sounds like you might be looking for a remote debugger. There are a couple of options:

• rconsole - This isn't really a debugger, but it allows you to get an interactive prompt inside another process. This can be useful for debugging purposes. I haven't tried this, but it looks relatively straightforward.
• rpdb2's embedded debugger - This lets you start a debugger and then connect to it from another shell.

Answer 2

At the moment the exception is caught, before the stack is unwound, the state is available for inspection with the inspect module: http://docs.python.org/2/library/inspect.html

Generically you would use inspect.getinnerframes on your traceback object. The local variables in each stack frame are available as .f_locals so you can see what they are.

The hard part is serializing all of them correctly: depending on what types you have in the local scope you may or may not be able to pickle them, dump them to JSON, or whatever.

Answer 3

I hope this helps (it helped me):

import logging, traceback
_logger = logging.getLogger(__name__)

try:
    something_bad()
except Exception as error:
    _logger.exception("Oh no!") # Logs original traceback
    store_exception_somewhere(error)

Also, in Python 3 there are a few new options like raise new_exc from original_exc or raise OtherException(...).with_traceback(tb) .

Answer 4

您可以做的是使用twisted.python并将回溯写入文件，它会为您提供包含异常的精确回溯

Answer 5

Well, as far as I know there is no easy solution to this problem. You might want to try this approach: How do I find what is using memory in a Python process in a production system?

Answer 6

You could create an entirely separate execution environment in the top level:

myEnv = {}
myEnv.update(globals)

Then execute your code within that execution environment. If an exception occurs you have the traceback (stack) and all the globals, so you can pretty well reconstruct the program state.