gradually warm java application in k8s with istio

Question

I have JVM-based applications that deliver excellent performance but need some time to “warm up” before reaching the top speed.

istio api (DestinationRule) does not support endpoint level weight, while envoy does.

I want a simple strategy: at the beginning when the pod is ready, the weight (qps) of that pod keeps a low level and increases over time

eg: lets I want the warm-up time to be 120 seconds. 2 pods have 50%-50% traffic, and when a new pod launches it will handle low traffic and increase over time and will reach 33% in 120 sec.

Let's take an example I have one deployment where 2 pods are running and I'm going to deploy a new version for my app when a new pod came traffic is equally divided by 33% each into 3 pods the new pods are unable to handle the traffic because the application needs some time to warm up and initiate the thread to handle the heavy load the same issue is coming during HPA (horizontal pod scaling).

I need a solution to warm up the pod and handle traffic gradually for new pods based on time duration

Answer 1

If you are using Kube.netes Ingress controller, then you can use the Peak EWMA load balancing mechanism.

You need add annotation for you ingress resource:

nginx.ingress.kube.netes.io/load-balance: ewma

See docs:

https://kube.netes.github.io/ingress-nginx/user-guide/nginx-configuration/configmap/#load-balance