Python Threaded Timer Returning Random Errors

Question

I have a python thread that runs every 20 seconds. The code is:

import threading
def work():
    Try:
        #code here
    except (SystemExit, KeyboardInterrupt):
        raise
    except Exception, e:
        logger.error('error somewhere',exc_info=True)    

    threading.Timer(20, work).start ();

It usually runs completely fine. Once in a while, it'll return an error that doesnt make much sense. The errors are the same two errors. The first one might be legitimate, but the errors after that definitely aren't. Then after that, it returns that same error every time it runs the thread. If I kill the process and start over, then it runs cleanly. I have absolutely no idea what going on here. Help please.

Answer 1

As currently defined in your question, you are most likely exceeding your maximum recursion depth. I can't be certain because you have omitted any opportunities for flow control that may be evident in your try block. Furthermore, everytime your code fails to execute, the general catch for exceptions will log the exception and then bump you into a new timer with a new logger (assume you are declaring that in the try block). I think you probably meant to do the following:

import threading
import time
def work():
    try:
        #code here
        pass
    except (SystemExit, KeyboardInterrupt):
        raise
    except Exception, e:
        logger.error('error somewhere',exc_info=True)    

t = threading.Timer(20, work)
t.start()
i = 0
while True:
    time.sleep(1)
    i+=1
    if i >1000:
        break
t.cancel()

If this is in fact the case, the reason your code was not working is that when you call your work function the first time, it processes and then right at the end, starts another work function in a new timer. This happens add infinitum until the stack fills up, python coughs, and gets angry that you have recursed (called a function from within itself) too many times.

My code fix pulls the timer outside of the function so we create a single timer, which calls the work function once every 20 seconds.

Because threading.timers run in separate threads, we also need to wait around in the main thread. To do this, I added a simple while loop that will run for 1000 seconds and then close the timer and exit. If we didn't wait around in the main loop, it would call your timer and then close out immediately causing python to clean up the timer before it executed even once.