Nginx 入口在 Kubernetes 上返回 502 Bad Gateway

Question

I have a Kubernetes cluster deployed on AWS (EKS). I deployed the cluster using the “eksctl” command line tool. I'm trying to deploy a Dash python app on the cluster without success. The default port for Dash is 8050. For the deployment I used the following resources:

• pod
• service (ClusterIP type)
• ingress

You can check the resource configuration files below:

pod-configuration-file.yml

kind: Pod
apiVersion: v1
metadata:
  name: dashboard-app
  labels:
    app: dashboard
spec:
  containers:
    - name: dashboard
      image: my_image_from_ecr
      ports:
        - containerPort: 8050

service-configuration-file.yml

kind: Service
apiVersion: v1
metadata:
  name: dashboard-service
spec:
  selector:
    app: dashboard
  ports:
    - port: 8050 # exposed port
      targetPort: 8050

ingress-configuration-file.yml (host based routing)

kind: Ingress
metadata:
  name: dashboard-ingress
  annotations:
    nginx.ingress.kubernetes.io/ssl-redirect: "false"
    nginx.ingress.kubernetes.io/force-ssl-redirect: "false"
    nginx.ingress.kubernetes.io/rewrite-target: /
spec:
  rules:
  - host: dashboard.my_domain
    http:
      paths:
      - backend:
          serviceName: dashboard-service
          servicePort: 8050
        path: /

I followed the steps below:

kubectl apply -f pod-configuration-file.yml
kubectl apply -f service-configuration-file.yml
kubectl apply -f ingress-confguration-file.yml

I also noticed that the pod deployment works as expected:

kubectl logs my_pod:

and the output is:

Dash is running on http://127.0.0.1:8050/

 Warning: This is a development server. Do not use app.run_server
 in production, use a production WSGI server like gunicorn instead.

 * Serving Flask app "annotation_analysis" (lazy loading)
 * Environment: production
   WARNING: This is a development server. Do not use it in a production deployment.
   Use a production WSGI server instead.
 * Debug mode: on

You can see from the ingress configuration file that I want to do host based routing using my domain. For this to work, I have also deployed an nginx-ingress. I have also created an “A” record set using Route53 that maps the “dashboard.my_domain” to the nginx-ingress:

kubectl get ingress

and the output is:

NAME                           HOSTS                                            ADDRESS.           PORTS.   AGE                                                            
dashboard-ingress         dashboard.my_domain                  nginx-ingress.elb.aws-region.amazonaws.com   80      93s

Moreover,

kubectl describe ingress dashboard-ingress

and the output is:

Name:             dashboard-ingress
Namespace:        default
Address:          nginx-ingress.elb.aws-region.amazonaws.com
Default backend:  default-http-backend:80 (<error: endpoints "default-http-backend" not found>)
Rules:
  Host                                            Path  Backends
  ----                                            ----  --------
  host.my-domain  
                                              /   dashboard-service:8050 (192.168.36.42:8050)
Annotations:                                      
nginx.ingress.kubernetes.io/force-ssl-redirect: false
                                              
nginx.ingress.kubernetes.io/rewrite-target: /
                                              
nginx.ingress.kubernetes.io/ssl-redirect: false
Events:                                           <none>

Unfortunately, when I try to access the Dash app on the browser, I get a 502 Bad Gateway error from the nginx. Could you please help me because my Kubernetes knowledge is limited. Thanks in advance.

Answer 1

It had nothing to do with Kubernetes or AWS settings. I had to change my python Dash code from:

if __name__ == "__main__":
    app.run_server(debug=True)

to:

if __name__ == "__main__":
    app.run_server(host='0.0.0.0',debug=True).

The addition of host='0.0.0.0' did the trick!

Answer 2

I think you'll need to check whether any other service is exposed at path / on the same host.

Secondly, try removing rewrite-target annotation. Also can you please update your question with output of kubectl describe ingress <ingress_Name>

I would also suggest you to use backend-protocol annotation with value as HTTP (default value, you can avoid using this if dashboard application is not SSL Configured, and only this application is served at the said host.) But, you may need to add this if multiple applications are served at this host, and create one Ingress with backend-protocol: HTTP for non SSL services, and another with backend-protocol: HTTPS to serve traffic to SSL enabled services.

For more information on backend-protocol annotation, kindly refer this link.

I have often faced this issue in my Ingress Setup and these steps have helped me resolve it.