Flask: making a non-blocking requests call

Question

Flask==2.0 (+uwsgi processes=5, threads=20) Python: 3.9

I have a route that will accept a message and re-direct this message to other API(s) based on the type of message. Let's say there are 4 types of messages with 4 matching downstream APIs. The request is made using requests library and downstream API will return a response that my route will need to return to the client (so I need to wait for the response).

The issue is sometimes one of the downstream APIs can have an issue exhibiting high latency. This has an undesired result as this makes my application slow and impact other message types.

Is there a way to make these requests calls non-blocking so the slow downstream API responses don't slow down the whole app? I read the flask async guide and from my understanding you can get real benefit if you're making multiple requests calls within a single route which isn't the case for me, it's always a single request.

Answer 1

Async requests to your downstream API could provide some relief in your case, I believe:

Async is beneficial when performing concurrent IO-bound tasks, but will probably not improve CPU-bound tasks.

Consider also using a queueing mechanism (such as background tasks in flask, or something more fully fledged such as RabbitMQ).

Final words - also consider defining timeout for your requests.

https://flask.palletsprojects.com/en/2.1.x/async-await/