Kubernetes Ingress nodejs timeout issue

Question

I am running my nodejs server on kubernetes cluster using Ingress. I am using below to prevent timeouts.

nginx.ingress.kubernetes.io/proxy-connect-timeout: "120"
nginx.ingress.kubernetes.io/proxy-read-timeout: "180"
nginx.ingress.kubernetes.io/proxy-send-timeout: "180"

Still i get 502 bad gateway error . Its not consistent, i get 502 at 7 sec, 20 sec 60 sec etc.

When i see Ingress logs, i see below errors

 shm_add_upstream::shm_add_node(host:port)failed while logging request

shm_add_node::ngx_slab_alloc_locked() failed: used_size[6313245], used_node[2542] while logging request, 

shm_add_server() failed while logging request

Is there a way to fix above issue? Is it related to any memory issue?

Answer 1

1. Check the shared memory size in use by nginx-module-vts

2. Set to more than 32M shared memory size by default.

   vhost_traffic_status_zone shared:vhost_traffic_status:32m

3. If the issue still appears, increase to more than (usedSize * 2).

Example nginx.conf

http {
  vhost_traffic_status_zone;
  vhost_traffic_status_zone shared:vhost_traffic_status:32m;
}