deployment, service, ingress implemented. Got default backend - 404
Question
I am practicing k8s deployment, service, and ingress implementation from here on GKE . Clicking through the simulator help me quickly understand concept, but when I do hands on. I stuck default backend - 404 . Here are my manifest files and bash description.
deployment.yaml
apiVersion: apps/v1beta2
kind: Deployment
metadata:
name: tunnel-deployment
labels:
app: tunnel
spec:
replicas: 1
selector:
matchLabels:
app: tunnel
template:
metadata:
labels:
app: tunnel
spec:
containers:
- name: tunnel
image: gcr.io/k8s-v1-235608/tunnel:latest
imagePullPolicy: Always
ports:
- containerPort: 8080
env:
- name: MONGODB_HOST
value: moon-mongodb-replicaset-client
- name: RABBIT_HOST
value: rodent-rabbitmq-headless
- name: RABBIT_PASSWORD
valueFrom:
secretKeyRef:
name: rodent-rabbitmq
key: rabbitmq-password
- name: REDIS_HOST
value: ninja-redis-ha
readinessProbe:
tcpSocket:
port: 8080
initialDelaySeconds: 5
periodSeconds: 10
livenessProbe:
tcpSocket:
port: 8080
initialDelaySeconds: 15
periodSeconds: 20
See deployments in bash
$ kubectl get deployments -o wide
NAME DESIRED CURRENT UP-TO-DATE AVAILABLE AGE CONTAINERS IMAGES SELECTOR
tunnel-deployment 1 1 1 1 11m tunnel gcr.io/k8s-v1-235608/tunnel:latest app=tunnel
See po in bash
$ kubectl get po -o wide
NAME READY STATUS RESTARTS AGE IP NODE NOMINATED NODE
bbox 1/1 Running 0 1d 10.4.0.6 gke-doom-default-pool-4b763b09-pxnz <none>
moon-mongodb-replicaset-0 1/1 Running 0 1d 10.4.2.9 gke-doom-default-pool-4b763b09-lcs6 <none>
moon-mongodb-replicaset-1 1/1 Running 0 1d 10.4.0.8 gke-doom-default-pool-4b763b09-pxnz <none>
moon-mongodb-replicaset-2 1/1 Running 0 1d 10.4.2.10 gke-doom-default-pool-4b763b09-lcs6 <none>
ninja-redis-ha-server-0 2/2 Running 0 1d 10.4.0.9 gke-doom-default-pool-4b763b09-pxnz <none>
ninja-redis-ha-server-1 2/2 Running 0 1d 10.4.1.11 gke-doom-default-pool-4b763b09-85ch <none>
ninja-redis-ha-server-2 2/2 Running 0 1d 10.4.2.11 gke-doom-default-pool-4b763b09-lcs6 <none>
rodent-rabbitmq-0 1/1 Running 0 1d 10.4.2.12 gke-doom-default-pool-4b763b09-lcs6 <none>
tunnel-deployment-fddf78dcc-lpq8l 1/1 Running 0 11m 10.4.1.37 gke-doom-default-pool-4b763b09-85ch <none>
service.yaml . I use NodePort with my tunnel-service . With my selection.app
apiVersion: v1
kind: Service
metadata:
name: tunnel-service
labels:
app: tunnel
spec:
type: NodePort
ports:
- name: tunnel-port
port: 80
targetPort: 8080
selector:
app: tunnel
Then I have single IP address to represent my service .
$ kubectl describe svc tunnel-service
Name: tunnel-service
Namespace: default
Labels: app=tunnel
Annotations: kubectl.kubernetes.io/last-applied-configuration:
{"apiVersion":"v1","kind":"Service","metadata":{"annotations":{},"labels":{"app":"tunnel"},"name":"tunnel-service","namespace":"default"},...
Selector: app=tunnel
Type: NodePort
IP: 10.7.242.110
Port: tunnel-port 80/TCP
TargetPort: 8080/TCP
NodePort: tunnel-port 32713/TCP
Endpoints: 10.4.1.37:8080
Session Affinity: None
External Traffic Policy: Cluster
Events: <none>
ingress.yaml . I route the / to my tunnel-service
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
name: tunnel-ingress
spec:
rules:
- host: abc.hbot.io
http:
paths:
- path: /
backend:
serviceName: tunnel-service
servicePort: 80
$ kubectl describe ingress
Name: tunnel-ingress
Namespace: default
Address: 35.244.186.216
Default backend: default-http-backend:80 (10.4.1.9:8080)
Rules:
Host Path Backends
---- ---- --------
abc.hbot.io
/ tunnel-service:80 (<none>)
Annotations:
ingress.kubernetes.io/backends: {"k8s-be-31768--d2232907436d0807":"HEALTHY","k8s-be-32713--d2232907436d0807":"UNHEALTHY"}
ingress.kubernetes.io/forwarding-rule: k8s-fw-default-tunnel-ingress--d2232907436d0807
ingress.kubernetes.io/target-proxy: k8s-tp-default-tunnel-ingress--d2232907436d0807
ingress.kubernetes.io/url-map: k8s-um-default-tunnel-ingress--d2232907436d0807
kubectl.kubernetes.io/last-applied-configuration: {"apiVersion":"extensions/v1beta1","kind":"Ingress","metadata":{"annotations":{},"name":"tunnel-ingress","namespace":"default"},"spec":{"rules":[{"host":"abc.hbot.io","http":{"paths":[{"backend":{"serviceName":"tunnel-service","servicePort":80},"path":"/"}]}}]}}
Events:
Type Reason Age From Message
---- ------ ---- ---- -------
Normal ADD 16m loadbalancer-controller default/tunnel-ingress
Normal CREATE 15m loadbalancer-controller ip: 35.244.186.216
In my browser. I had routed my hostname to my correct DNS and got default backend - 404
I confirm my pod in deployment works. My endpoint does response on my request.
bash-4.3# curl http://localhost:8080/api
Hi I'm API Interface
Update on Attempt #1 :
Add * after / and apply
$ kubectl apply -f ingress.yaml
ingress.extensions/tunnel-ingress configured
$ kubectl describe ingress
Name: tunnel-ingress
Namespace: default
Address: 35.244.186.216
Default backend: default-http-backend:80 (10.4.1.9:8080)
Rules:
Host Path Backends
---- ---- --------
abc.hbot.io
/* tunnel-service:80 (<none>)
Annotations:
ingress.kubernetes.io/backends: {"k8s-be-31768--d2232907436d0807":"HEALTHY","k8s-be-32713--d2232907436d0807":"UNHEALTHY"}
ingress.kubernetes.io/forwarding-rule: k8s-fw-default-tunnel-ingress--d2232907436d0807
ingress.kubernetes.io/target-proxy: k8s-tp-default-tunnel-ingress--d2232907436d0807
ingress.kubernetes.io/url-map: k8s-um-default-tunnel-ingress--d2232907436d0807
kubectl.kubernetes.io/last-applied-configuration: {"apiVersion":"extensions/v1beta1","kind":"Ingress","metadata":{"annotations":{},"name":"tunnel-ingress","namespace":"default"},"spec":{"rules":[{"host":"abc.hbot.io","http":{"paths":[{"backend":{"serviceName":"tunnel-service","servicePort":80},"path":"/*"}]}}]}}
Events:
Type Reason Age From Message
---- ------ ---- ---- -------
Normal ADD 42m loadbalancer-controller default/tunnel-ingress
Normal CREATE 40m loadbalancer-controller ip: 35.244.186.216
Not work.
Attempt #2
Add annotations
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
name: tunnel-ingress
annotations:
kubernetes.io/ingress.class: "nginx"
spec:
rules:
- host: abc.hbot.io
http:
paths:
- path: /
backend:
serviceName: tunnel-service
servicePort: 80
Check the ingress
$ kubectl describe ingress
Name: tunnel-ingress
Namespace: default
Address: 35.244.186.216
Default backend: default-http-backend:80 (10.4.1.9:8080)
Rules:
Host Path Backends
---- ---- --------
abc.hbot.io
/ tunnel-service:80 (<none>)
Annotations:
ingress.kubernetes.io/backends: {"k8s-be-31768--d2232907436d0807":"HEALTHY","k8s-be-32713--d2232907436d0807":"UNHEALTHY"}
ingress.kubernetes.io/forwarding-rule: k8s-fw-default-tunnel-ingress--d2232907436d0807
ingress.kubernetes.io/target-proxy: k8s-tp-default-tunnel-ingress--d2232907436d0807
ingress.kubernetes.io/url-map: k8s-um-default-tunnel-ingress--d2232907436d0807
kubectl.kubernetes.io/last-applied-configuration: {"apiVersion":"extensions/v1beta1","kind":"Ingress","metadata":{"annotations":{"kubernetes.io/ingress.class":"nginx"},"name":"tunnel-ingress","namespace":"default"},"spec":{"rules":[{"host":"abc.hbot.io","http":{"paths":[{"backend":{"serviceName":"tunnel-service","servicePort":80},"path":"/"}]}}]}}
kubernetes.io/ingress.class: nginx
Events:
Type Reason Age From Message
---- ------ ---- ---- -------
Normal ADD 54m loadbalancer-controller default/tunnel-ingress
Normal CREATE 53m loadbalancer-controller ip: 35.244.186.216
Error: Server Error The server encountered a temporary error and could not complete your request. Please try again in 30 seconds.
Attemp #3
Install ingress-nginx to cluster. I follow the cert-manager doc . Because I used it once with former cluster.
Follow the same name.
$ helm install stable/nginx-ingress --name quickstart
NAME: quickstart
LAST DEPLOYED: Wed Mar 27 16:09:15 2019
NAMESPACE: default
STATUS: DEPLOYED
RESOURCES:
==> v1/ConfigMap
NAME DATA AGE
quickstart-nginx-ingress-controller 1 0s
==> v1/Pod(related)
NAME READY STATUS RESTARTS AGE
quickstart-nginx-ingress-controller-5b8d54d964-rnvw6 0/1 ContainerCreating 0 0s
quickstart-nginx-ingress-default-backend-57bdfdcd46-vtf4h 0/1 Pending 0 0s
==> v1/Service
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
quickstart-nginx-ingress-controller LoadBalancer 10.7.241.190 <pending> 80:32341/TCP,443:32762/TCP 0s
quickstart-nginx-ingress-default-backend ClusterIP 10.7.254.207 <none> 80/TCP 0s
==> v1/ServiceAccount
NAME SECRETS AGE
quickstart-nginx-ingress 1 0s
==> v1beta1/ClusterRole
NAME AGE
quickstart-nginx-ingress 0s
==> v1beta1/ClusterRoleBinding
NAME AGE
quickstart-nginx-ingress 0s
==> v1beta1/Deployment
NAME READY UP-TO-DATE AVAILABLE AGE
quickstart-nginx-ingress-controller 0/1 1 0 0s
quickstart-nginx-ingress-default-backend 0/1 1 0 0s
==> v1beta1/Role
NAME AGE
quickstart-nginx-ingress 0s
==> v1beta1/RoleBinding
NAME AGE
quickstart-nginx-ingress 0s
NOTES:
The nginx-ingress controller has been installed.
It may take a few minutes for the LoadBalancer IP to be available.
You can watch the status by running 'kubectl --namespace default get services -o wide -w quickstart-nginx-ingress-controller'
An example Ingress that makes use of the controller:
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
annotations:
kubernetes.io/ingress.class: nginx
name: example
namespace: foo
spec:
rules:
- host: www.example.com
http:
paths:
- backend:
serviceName: exampleService
servicePort: 80
path: /
# This section is only required if TLS is to be enabled for the Ingress
tls:
- hosts:
- www.example.com
secretName: example-tls
If TLS is enabled for the Ingress, a Secret containing the certificate and key must also be provided:
apiVersion: v1
kind: Secret
metadata:
name: example-tls
namespace: foo
data:
tls.crt: <base64 encoded cert>
tls.key: <base64 encoded key>
type: kubernetes.io/tls
describe ingress
$ kubectl describe ingress
Name: tunnel-ingress
Namespace: default
Address: 35.244.186.216
Default backend: default-http-backend:80 (10.4.1.9:8080)
Rules:
Host Path Backends
---- ---- --------
abc.hbot.io
/ tunnel-service:80 (<none>)
Annotations:
ingress.kubernetes.io/backends: {"k8s-be-31768--d2232907436d0807":"HEALTHY","k8s-be-32713--d2232907436d0807":"UNHEALTHY"}
ingress.kubernetes.io/forwarding-rule: k8s-fw-default-tunnel-ingress--d2232907436d0807
ingress.kubernetes.io/target-proxy: k8s-tp-default-tunnel-ingress--d2232907436d0807
ingress.kubernetes.io/url-map: k8s-um-default-tunnel-ingress--d2232907436d0807
kubectl.kubernetes.io/last-applied-configuration: {"apiVersion":"extensions/v1beta1","kind":"Ingress","metadata":{"annotations":{"kubernetes.io/ingress.class":"nginx"},"name":"tunnel-ingress","namespace":"default"},"spec":{"rules":[{"host":"abc.hbot.io","http":{"paths":[{"backend":{"serviceName":"tunnel-service","servicePort":80},"path":"/"}]}}]}}
kubernetes.io/ingress.class: nginx
Events:
Type Reason Age From Message
---- ------ ---- ---- -------
Normal CREATE 5m12s nginx-ingress-controller Ingress default/tunnel-ingress
Normal CREATE 7s (x7 over 92m) loadbalancer-controller ip: 35.244.186.216
Normal UPDATE 7s (x13 over 5m12s) nginx-ingress-controller Ingress default/tunnel-ingress
Test with curl on my laptop
$ curl -H 'Host: abc.hbot.io' 35.244.186.216/api
<html><head>
<meta http-equiv="content-type" content="text/html;charset=utf-8">
<title>502 Server Error</title>
</head>
<body text=#000000 bgcolor=#ffffff>
<h1>Error: Server Error</h1>
<h2>The server encountered a temporary error and could not complete your request.<p>Please try again in 30 seconds.</h2>
<h2></h2>
</body></html>
Attempt 4th:
Solve insufficient cpu otherwise my new pods will not run values.yaml
controller:
resources:
limits:
cpu: 100m
memory: 64Mi
requests:
cpu: 100m
memory: 64Mi
defaultBackend:
resources:
limits:
cpu: 10m
memory: 20Mi
requests:
cpu: 10m
memory: 20Mi
Install ingress-nginx helm install --values values.yaml stable/nginx-ingress --name quickstart
ingress.yaml
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
name: tunnel-ingress
annotations:
kubernetes.io/ingress.class: nginx
spec:
rules:
- host: abc.hbot.io
http:
paths:
- backend:
serviceName: tunnel-service
servicePort: 80
path: /
describe ingress
$ kubectl describe ingress
Name: tunnel-ingress
Namespace: default
Address: 35.240.162.185
Default backend: default-http-backend:80 (10.4.1.9:8080)
Rules:
Host Path Backends
---- ---- --------
abc.hbot.io
/ tunnel-service:80 (<none>)
Annotations:
kubectl.kubernetes.io/last-applied-configuration: {"apiVersion":"extensions/v1beta1","kind":"Ingress","metadata":{"annotations":{"kubernetes.io/ingress.class":"nginx"},"name":"tunnel-ingress","namespace":"default"},"spec":{"rules":[{"host":"abc.hbot.io","http":{"paths":[{"backend":{"serviceName":"tunnel-service","servicePort":80},"path":"/"}]}}]}}
kubernetes.io/ingress.class: nginx
Events:
Type Reason Age From Message
---- ------ ---- ---- -------
Normal CREATE 14m nginx-ingress-controller Ingress default/tunnel-ingress
Normal CREATE 14m nginx-ingress-controller Ingress default/tunnel-ingress
Normal UPDATE 14m nginx-ingress-controller Ingress default/tunnel-ingress
Normal CREATE 9m27s nginx-ingress-controller Ingress default/tunnel-ingress
I delete and apply ingress again. IP address is changed
$ kubectl get ingress -o wide
NAME HOSTS ADDRESS PORTS AGE
tunnel-ingress abc.hbot.io 35.240.162.185 80 15m
My situation is even worse.
$ curl -H 'Host: abc.hbot.io' http://35.240.162.185/api
curl: (7) Failed to connect to 35.240.162.185 port 80: Connection refused
Final Attempt #5:
I gave up using plain http setup. And start over with https approach. By following cert-manager and then replace kuard service by my service. Then it is done!
Question:
Where am I wrong?
1 answers
solution1
0 2019-03-27 08:22:16
I never used GKE, but with AWS and ALB, the path needs to include the wildcard. It seems that it's the same on GKE. https://cloud.google.com/kubernetes-engine/docs/concepts/ingress#multiple_backend_services https://cloud.google.com/kubernetes-engine/docs/tutorials/http-balancer#step_6_optional_serving_multiple_applications_on_a_load_balancer
Try to configure your ingress as follows:
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
name: tunnel-ingress
spec:
rules:
- host: abc.hbot.io
http:
paths:
- path: /*
backend:
serviceName: tunnel-service
servicePort: 80
The technical post webpages of this site follow the CC BY-SA 4.0 protocol. If you need to reprint, please indicate the site URL or the original address.Any question please contact:yoyou2525@163.com.
Related Question
Google Kubernetes Engine Ingress UNHEALTHY backend service
Ingress unable to find backend service in Kubernetes v1.21.5
kubernetes unhealthy ingress backend
Kubernetes ingress returning 404
ECS fargate deployment got stuck and failed to deploy ECS Service
Kubernetes ingress-nginx call service from non-default namespace
java.sql.SQLFeatureNotSupportedException: Method org.postgresql.jdbc4.Jdbc4Connection.isValid(int) is not yet implemented. in Spring Boot
How to expose a service using Ingress?
Nginx Ingress Controller Returns 404 Kubernetes
How to modify default docker base image during deployment of Azure Kubernetes service
Related Tags