Nginx-Ingress-Controller limit number of rps to deployment

Question

I am having issues serving data under high load and would like to return a 503 if the total rps to a specific deployment exceeds a certain number. I was playing around with their rate limiting but that limits per client.

Thanks!

Answer 1

Simpler approach, I would try, would be to limit the resources (CPU and Memory) for that deployment. You can also separate the ingress controller for that service and limit resources for that as well.

Alternatively service meshes provide more customizations, for example istio provides ConnectionPoolSettings for DestinationRule objects that can serve similar purpose.

Additionally, there is also an experimental feature of traffic shaping that might be helpful but that requires updating your CNI configuration.

---

For reference: @Dylan figured out that limit_req_zone Nginx configuration can be used with $host to achieve this as well. In order to do this, (1) update your nginx ingress controller config map and add the http-snippet

data:
  http-snippet: |
    limit_req_zone $host zone=queue:10m rate=1r/s;

You may have to restart your controller pods to pick up the new configuration. (2) Apply the configuration to the Ingress using configuration-snippet :

apiVersion: extensions/v1beta1
kind: Ingress
metadata:
  annotations:
    nginx.ingress.kubernetes.io/configuration-snippet: |
      limit_req zone=queue burst=12 delay=8;
...