In GKE, how to minimize connect time with Load balancer

Question

In GKE, for cost-saving, I usually put the node number to zero. When I autoscale nodes(or say add) and run the pods. It takes more than 6-7 mins to connect to Loadbalancer and up the URL. That's why health checks in the waiting state. Is there any way to reduce the time? Thanks

Answer 1

In short not really. Spinning up time of nodes is not easily controlled, basically it is the time that will take for the VM to be allocated, turned on, boot the OS and do some other stuff related to Kubernetes (like configuration, adding to node pool, etc) this takes time. In addition to Pods spinning up time which depends on the Docker image (size/dependencies etc).

Scaling down your application to zero nodes is not very recommended. It is always recommended to have some nodes up (don't you have other apps running on the GKE cluster? Kubernetes clusters are recommended to have at least 3 nodes running).

Have you considered using Cloud Functions? Is it possible in your case? This the the best option I know of for a quick scale up and zero scale down. And in general you can keep some kind of “ping” to the function to keep it “hot” for a relatively cheap price.

If none of the options above is possible (id say keeping your node pool with at least 3 nodes operating, is best as it is takes time for the Kubernetes control plan to boot). I suggest starting with reducing the spinning up time of your Pods by improving the Docker image - reducing its size etc.

Here are some articles on how to reduce Docker image size

https://phoenixnap.com/kb/docker-image-size

https://www.ardanlabs.com/blog/2020/02/docker-images-part1-reducing-image-size.html

After that I will experiment with different machine types for node to check which one is spinning the fastest - could be an interesting thing to do in any case

Here is an interesting comparison on VM spinning up times

https://www.google.com/amp/s/blog.cloud66.com/part-2-comparing-the-speed-of-vm-creation-and-ssh-access-on-aws-digitalocean-linode-vexxhost-google-cloud-rackspace-packet-cloud-a-and-microsoft-azure/amp/

Answer 2

If Cloud Functions is not an option, you might want to look at Cloud Run (which supports containers and scales to zero) or GKE Autopilot (which does not scale to zero, but you can scale down to low resource and it will autoscale up and down as needed)