[英]What is the difference between kubernetes.io/ingress.class: "nginx" and kubernetes.io/ingress.class: "private"
[英]ingress-nginx not working when using ingressClassName instead of kubernetes.io/ingress.class in annotations
我有一個使用 Kubespray 部署的裸機集群,啟用了 kubernetes 1.22.2、MetalLB 和 ingress-nginx。 在設置ingressClassName: nginx
時嘗試訪問通過 helm 部署的任何服務時,我收到“404 Not found”。 但是,當我不使用ingressClassName: nginx
並在注釋中使用kubernetes.io/ingress.class: nginx
而不是掌舵圖表 values.yaml 時,一切正常。 如何使用ingressClassName
讓它工作?
這些是我對inventory/mycluster/group_vars/k8s_cluster/addons.yml
/mycluster/group_vars/k8s_cluster/addons.yml 的 kubespray 設置
# Nginx ingress controller deployment
ingress_nginx_enabled: true
ingress_nginx_host_network: false
ingress_publish_status_address: ""
ingress_nginx_nodeselector:
kubernetes.io/os: "linux"
ingress_nginx_tolerations:
- key: "node-role.kubernetes.io/master"
operator: "Equal"
value: ""
effect: "NoSchedule"
- key: "node-role.kubernetes.io/control-plane"
operator: "Equal"
value: ""
effect: "NoSchedule"
ingress_nginx_namespace: "ingress-nginx"
ingress_nginx_insecure_port: 80
ingress_nginx_secure_port: 443
ingress_nginx_configmap:
map-hash-bucket-size: "128"
ssl-protocols: "TLSv1.2 TLSv1.3"
ingress_nginx_configmap_tcp_services:
9000: "default/example-go:8080"
ingress_nginx_configmap_udp_services:
53: "kube-system/coredns:53"
ingress_nginx_extra_args:
- --default-ssl-certificate=default/mywildcard-tls
ingress_nginx_class: "nginx"
grafana helm values.yaml
ingress:
enabled: true
# For Kubernetes >= 1.18 you should specify the ingress-controller via the field ingressClassName
# See https://kubernetes.io/blog/2020/04/02/improvements-to-the-ingress-api-in-kubernetes-1.18/#specifying-the-class-of-an-ingress
ingressClassName: nginx
# Values can be templated
annotations:
# kubernetes.io/ingress.class: nginx
# kubernetes.io/tls-acme: "true"
labels: {}
path: /
# pathType is only for k8s >= 1.1=
pathType: Prefix
hosts:
- grafana.mycluster.org
tls:
- secretName: mywildcard-tls
hosts:
- grafana.mycluster.org
kubectl describe pod grafana-679bbfd94-p2dd7
...
Events:
Type Reason Age From Message
---- ------ ---- ---- -------
Normal Scheduled 25m default-scheduler Successfully assigned default/grafana-679bbfd94-p2dd7 to node1
Normal Pulled 25m kubelet Container image "grafana/grafana:8.2.2" already present on machine
Normal Created 25m kubelet Created container grafana
Normal Started 25m kubelet Started container grafana
Warning Unhealthy 24m (x3 over 25m) kubelet Readiness probe failed: Get "http://10.233.90.33:3000/api/health": dial tcp 10.233.90.33:3000: connect: connection refused
kubectl get svc
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
grafana LoadBalancer 10.233.14.90 10.10.30.52 80:30285/TCP 55m
kubernetes ClusterIP 10.233.0.1 <none> 443/TCP 9d
kubectl get ing
(未分配節點地址)
NAME CLASS HOSTS ADDRESS PORTS AGE
grafana nginx grafana.mycluster.org 80, 443 25m
kubectl describe ing grafana
(未分配節點地址)
Name: grafana
Namespace: default
Address:
Default backend: default-http-backend:80 (<error: endpoints "default-http-backend" not found>)
TLS:
mywildcard-tls terminates grafana.mycluster.org
Rules:
Host Path Backends
---- ---- --------
grafana.mycluster.org
/ grafana:80 (10.233.90.33:3000)
Annotations: meta.helm.sh/release-name: grafana
meta.helm.sh/release-namespace: default
Events: <none>
kubectl get all --all-namespaces
NAMESPACE NAME READY STATUS RESTARTS AGE
default pod/grafana-b988b9b6-pxccw 1/1 Running 0 2m53s
default pod/nfs-client-nfs-subdir-external-provisioner-68f44cd9f4-wjlpv 1/1 Running 0 17h
ingress-nginx pod/ingress-nginx-controller-6m2vt 1/1 Running 0 17h
ingress-nginx pod/ingress-nginx-controller-xkgxl 1/1 Running 0 17h
kube-system pod/calico-kube-controllers-684bcfdc59-kmsst 1/1 Running 0 17h
kube-system pod/calico-node-dhlnt 1/1 Running 0 17h
kube-system pod/calico-node-r8ktz 1/1 Running 0 17h
kube-system pod/coredns-8474476ff8-9sbwh 1/1 Running 0 17h
kube-system pod/coredns-8474476ff8-fdgcb 1/1 Running 0 17h
kube-system pod/dns-autoscaler-5ffdc7f89d-vskvq 1/1 Running 0 17h
kube-system pod/kube-apiserver-node1 1/1 Running 0 17h
kube-system pod/kube-controller-manager-node1 1/1 Running 1 17h
kube-system pod/kube-proxy-hbjz6 1/1 Running 0 16h
kube-system pod/kube-proxy-lfqzt 1/1 Running 0 16h
kube-system pod/kube-scheduler-node1 1/1 Running 1 17h
kube-system pod/kubernetes-dashboard-548847967d-qqngw 1/1 Running 0 17h
kube-system pod/kubernetes-metrics-scraper-6d49f96c97-2h7hc 1/1 Running 0 17h
kube-system pod/nginx-proxy-node2 1/1 Running 0 17h
kube-system pod/nodelocaldns-64cqs 1/1 Running 0 17h
kube-system pod/nodelocaldns-t5vv6 1/1 Running 0 17h
kube-system pod/registry-proxy-kljvw 1/1 Running 0 17h
kube-system pod/registry-proxy-nz4qk 1/1 Running 0 17h
kube-system pod/registry-xzh9d 1/1 Running 0 17h
metallb-system pod/controller-77c44876d-c92lb 1/1 Running 0 17h
metallb-system pod/speaker-fkjqp 1/1 Running 0 17h
metallb-system pod/speaker-pqjgt 1/1 Running 0 17h
NAMESPACE NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
default service/grafana LoadBalancer 10.233.1.104 10.10.30.52 80:31116/TCP 2m53s
default service/kubernetes ClusterIP 10.233.0.1 <none> 443/TCP 17h
kube-system service/coredns ClusterIP 10.233.0.3 <none> 53/UDP,53/TCP,9153/TCP 17h
kube-system service/dashboard-metrics-scraper ClusterIP 10.233.35.124 <none> 8000/TCP 17h
kube-system service/kubernetes-dashboard ClusterIP 10.233.32.133 <none> 443/TCP 17h
kube-system service/registry ClusterIP 10.233.30.221 <none> 5000/TCP 17h
NAMESPACE NAME DESIRED CURRENT READY UP-TO-DATE AVAILABLE NODE SELECTOR AGE
ingress-nginx daemonset.apps/ingress-nginx-controller 2 2 2 2 2 kubernetes.io/os=linux 17h
kube-system daemonset.apps/calico-node 2 2 2 2 2 kubernetes.io/os=linux 17h
kube-system daemonset.apps/kube-proxy 2 2 2 2 2 kubernetes.io/os=linux 17h
kube-system daemonset.apps/nodelocaldns 2 2 2 2 2 kubernetes.io/os=linux 17h
kube-system daemonset.apps/registry-proxy 2 2 2 2 2 <none> 17h
metallb-system daemonset.apps/speaker 2 2 2 2 2 kubernetes.io/os=linux 17h
NAMESPACE NAME READY UP-TO-DATE AVAILABLE AGE
default deployment.apps/grafana 1/1 1 1 2m53s
default deployment.apps/nfs-client-nfs-subdir-external-provisioner 1/1 1 1 17h
kube-system deployment.apps/calico-kube-controllers 1/1 1 1 17h
kube-system deployment.apps/coredns 2/2 2 2 17h
kube-system deployment.apps/dns-autoscaler 1/1 1 1 17h
kube-system deployment.apps/kubernetes-dashboard 1/1 1 1 17h
kube-system deployment.apps/kubernetes-metrics-scraper 1/1 1 1 17h
metallb-system deployment.apps/controller 1/1 1 1 17h
NAMESPACE NAME DESIRED CURRENT READY AGE
default replicaset.apps/grafana-b988b9b6 1 1 1 2m53s
default replicaset.apps/nfs-client-nfs-subdir-external-provisioner-68f44cd9f4 1 1 1 17h
kube-system replicaset.apps/calico-kube-controllers-684bcfdc59 1 1 1 17h
kube-system replicaset.apps/coredns-8474476ff8 2 2 2 17h
kube-system replicaset.apps/dns-autoscaler-5ffdc7f89d 1 1 1 17h
kube-system replicaset.apps/kubernetes-dashboard-548847967d 1 1 1 17h
kube-system replicaset.apps/kubernetes-metrics-scraper-6d49f96c97 1 1 1 17h
kube-system replicaset.apps/registry 1 1 1 17h
metallb-system replicaset.apps/controller-77c44876d 1 1 1 17h
kubectl get ing grafana -o yaml
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
annotations:
kubernetes.io/tls-acme: "true"
meta.helm.sh/release-name: grafana
meta.helm.sh/release-namespace: default
nginx.ingress.kubernetes.io/force-ssl-redirect: "true"
nginx.ingress.kubernetes.io/ssl-redirect: "true"
creationTimestamp: "2021-11-11T07:16:12Z"
generation: 1
labels:
app.kubernetes.io/instance: grafana
app.kubernetes.io/managed-by: Helm
app.kubernetes.io/name: grafana
app.kubernetes.io/version: 8.2.2
helm.sh/chart: grafana-6.17.5
name: grafana
namespace: default
resourceVersion: "3137"
uid: 6c34d3bd-9ab6-42fe-ac1b-7620a9566f62
spec:
ingressClassName: nginx
rules:
- host: grafana.mycluster.org
http:
paths:
- backend:
service:
name: ssl-redirect
port:
name: use-annotation
path: /*
pathType: Prefix
- backend:
service:
name: grafana
port:
number: 80
path: /
pathType: Prefix
status:
loadBalancer: {}
運行
kubectl get ingressclass
返回“未找到資源”。
這是你問題的主要原因。
為什么?
當您在 Grafana values.yaml
文件中指定ingressClassName: nginx
,您將 Ingress 資源設置為使用不存在的nginx
Ingress 類。
我使用minikube 、 MetalLB和NGINX Ingress復制了您的問題,這些文件通過修改后的deploy.yaml 文件安裝,並帶有注釋的IngressClass
資源 + 將 NGINX Ingress 控制器名稱設置為nginx
如您的示例所示。 結果完全一樣 - ingressClassName: nginx
沒有工作(沒有地址),但注釋kubernetes.io/ingress.class: nginx
工作。
(對於下面的解決方案,我使用控制器 pod 名稱ingress-nginx-controller-86c865f5c4-qwl2b
,但在您的情況下它會有所不同 - 使用kubectl get pods -n ingress-nginx
命令檢查它。另外請記住它的種類一種解決方法 - 通常ingressClass
資源應該與 NGINX Ingress 的整個安裝一起自動安裝。我提出這個解決方案是為了了解為什么它以前對你不起作用,以及為什么它可以與使用 helm 安裝的 NGINX Ingress 一起使用)
在我發現的 Ingress NGINX 控制器的日志中( kubectl logs ingress-nginx-controller-86c865f5c4-qwl2b -n ingress-nginx
):
"Ignoring ingress because of error while validating ingress class" ingress="default/minimal-ingress" error="no object matching key \"nginx\" in local store"
所以它清楚地表明沒有與nginx
控制器類匹配的鍵 - 因為沒有ingressClass
資源,它是 NGINX Ingress 控制器和正在運行的 Ingress 資源之間的“鏈接”。
您可以通過運行kubectl get pod ingress-nginx-controller-86c865f5c4-qwl2b -n ingress-nginx -o yaml
來將控制器類的名稱提供給控制器:
...
spec:
containers:
- args:
- /nginx-ingress-controller
- --publish-service=$(POD_NAMESPACE)/ingress-nginx-controller
- --election-id=ingress-controller-leader
- --controller-class=k8s.io/nginx
...
現在我將創建並應用以下 Ingress 類資源:
apiVersion: networking.k8s.io/v1
kind: IngressClass
metadata:
name: nginx
spec:
controller: k8s.io/nginx
現在在日志中我可以看到它已正確配置:
I1115 12:13:42.410384 7 main.go:101] "successfully validated configuration, accepting" ingress="minimal-ingress/default"
I1115 12:13:42.420408 7 store.go:371] "Found valid IngressClass" ingress="default/minimal-ingress" ingressclass="nginx"
I1115 12:13:42.421487 7 event.go:282] Event(v1.ObjectReference{Kind:"Ingress", Namespace:"default", Name:"minimal-ingress", UID:"c708a672-a8dd-45d3-a2ec-f2e2881623ea", APIVersion:"networking.k8s.io/v1", ResourceVersion:"454362", FieldPath:""}): type: 'Normal' reason: 'Sync' Scheduled for sync
我重新應用了入口資源定義,我獲得了入口資源的 IP 地址。
正如我之前所說,我建議不要使用這種解決方法,而是建議使用自動安裝IngressClass
的解決方案來安裝 NGINX Ingress 資源。 當您選擇 helm chart 時,它具有Ingress Class資源,因此問題消失了。 其他可能的安裝方法在這里。
聲明:本站的技術帖子網頁,遵循CC BY-SA 4.0協議,如果您需要轉載,請注明本站網址或者原文地址。任何問題請咨詢:yoyou2525@163.com.