gevent, requests and unhandled exception

Question

I have code:

import gevent
import gevent.monkey; gevent.monkey.patch_all()
import requests

def func():
    try:
        requests.get('http://unavailable-host/')
    except:
        pass

def main():
    jobs = [gevent.spawn(func) for i in xrange(10)]
    gevent.joinall(jobs)

if __name__ == '__main__':
    main()

This script usually nothing output. But sometimes (in 1 of 5 runs) i get this message:

Unhandled exception in thread started by
sys.excepthook is missing
lost sys.stderr

Explain me, why this happen, and what is right solution? Also, if I add

gevent.sleep(1)

after

gevent.joinall(jobs)

script always nothing output, all is ok.

Answer 1

Edit:

This seems like it has to do with a thread trying to do stuff (like print to stdout/stderr) after the main program has already exited.

See Python Bugs: issue1722344 for reference,

and Martijn Pieters's comment in this answer to a similar SO question :

Indeed, the error is generated because python is exiting and there is still a thread active.

Previous (and completely incorrect) answer:

What you are experiencing is one of the not-so-fun side effects of monkey_patch .

requests uses socket as the underlying mechanism for transferring data across the internet to some address. gevent.monkey.patch_all() replaces the stdlib socket with gevent.socket , which is an asynchronous (non-blocking) socket, and so when somewhere deep within the code (my guess goes to http.client which is used by urllib , which is in turn used by requests ), where a sock.recv(X) command is made, where the code is expected to block until X bytes are received or the socket is closed , and instead, because the socket has been replaced by gevent.socket , which returns immediately with only as many bytes as are currently in the stream buffer , the code breaks.

The easy solution in your case, however, is to simply use grequests , which is requests built to use gevent (and which, in fact, does its own monkey_patch.)