debugging: how to check what where my Python program is hanging?

Question

A fairly large Python program I write, runs, but sometimes, after running for minutes or hours, in a non easily reproducible moment, hangs and outputs nothing to the screen.

I have no idea what it is doing at that moment, and in what part of code it is.

How can I run this in a debugger or something to see what lines of code is the program executing in the moment it hangs?

Its too large to put "print" statements all over the place.

I did:

python -m trace --trace /usr/local/bin/my_program.py

but that gives me so much output that I can't really see anything, just millions of lines scrolling on the screen.

Best would be if I could send some signal to the program with "kill -SIGUSR1" or something, and at that moment the program would drop into a debugger and show me the line it stopped at and possibly allow me to step through the program then.

I've tried:

pdb usr/local/bin/my_program.py

and then:

(Pdb) cont

but what do I do to see where I am when it hangs? It doesn't throw and exception, just seems like it waits for something, possibly in an infinite loop.

One more detail: when the program hangs, and I press ^C and then (not sure if that is necessary) the program continues normally (without throwing any exception and without giving me any hint on the screen why did it stop).

Answer 1

This could be useful to you. I usually do

>>> import pdb
>>> import program2debug
>>> pdb.run('program2debug.test()')

I usually add a -v option to my programs, which enables tons of print statements explaining what I'm doing in detail. When you write a program in the future, consider doing the same before it gets thousands of lines big.

Answer 2

You could try running it in debug mode in an IDE like pydev (eclipse) or pycharm. You can break the program at any moment and get to its current execution point.

Answer 3

No program is ever too big to put print statements all over the place. You need to read up on the logging module and insert lots of logging.debug() statements. This is just a better form of print statement that outputs to a file, and can be turned off easily in production software. But years from now, when you need to modify the code, you can easily turn it all back on and get the benefit of the insight of the original programmer.