Azure load balancer health probe failure

Question

I have gone through this https://docs.microsoft.com/en-us/azure/load-balancer/load-balancer-custom-probe-overview but i haven't found an answer

Problem: i have tensorflow applications running on individual VMs hosted by gunicorn + flask application. The intention is to ensure every VM gets only one request at a time. So we have configured our app in such a way that during a request being processed , if we receive another, we simply send a BUSY code back (non 200 response) ..now this fails the health probe BUT we have no idea when and how it adds this VM back to the pool since in reality , this VM was just busy and NOT in poor health..since azure LB doesn't understand application running on VMs we didn't know how else to solve this

But we are seeing a lot of timeouts, poor utilisation of existing VMs etc when we use the above approach, prompting us to wonder if the "poor health" guys are even being recalled..azure documentation and support is really poor ..any pointers please?

Answer 1

As per the documentation here Load balancer operates on layer 4 and doesn't provide application layer gateway functionality. You could try following steps to better understand the workflow and accordingly configure your LB for better efficiency.

• You can try and set-up Monitoring for your load balancer and go through some of the metrics like Flow Distribution . This view can give you feedback on whether your Load Balancer configuration or traffic patterns are leading to imbalanced traffic. For example, if you have session affinity configured and a single client is making a disproportionate number of requests.
• As mentioned in the documentation you have shared above, if a health probe fails the particular backend is marked as unhealthy and if the next health probe is successful the backend is marked as healthy (Unhealthy threshold mentioned by you also plays a role here). You can try and optimize the health probe intervals in order to maximize efficiency of your VMs.