Why does Eventlet patch fix requests.post in Celery task?

Question

I was having issues with Celery workers timing out due to multiple HTTP requests in my tasks. I didn't understand why. It appeared that the tasks were just getting stuck at the first requests.post line (although not always). After finding a tangential SO answer regarding requests, I patched the requests library with:

#import requests
import eventlet
requests = eventlet.import_patched("requests")

And everything is working very quickly and without issue now.

My question is: What voodoo sorcery is Eventlet doing to make my tasks work as expected?

Answer 1

I am assuming, you meant task timeout as in celery's time-limit or soft-time-limit. If it means requests library's timeout period then following answer does not apply.

If you are using eventlet as processpool then that patching has simply made your requests library behave as it is suppose to be behaving in a non-blocking execution environment. If you are using prefork as pocesspool, then it could be that you are not using the response from the requests.post call. In that case, the eventlet patching simply made your requests library non-blocking. Since it doesn't block anymore, your task can move on and hence you don't see the task timeout, regardless of the response/timeout happening on requests library level