與 dns 名稱的 Pod 間通信在 kubernetes 中不起作用

Question

我已經使用 kubeadm 在一台帶有 flannel(v0.11.0-amd64) 的服務器（不是 VM）上安裝了 Kubernetes（1.17.3）。 然后我安裝了 grafana 和 prometheus 並且可以在 NodePort http://<serverip>:31000上訪問兩者

現在，當我嘗試從 grafana 訪問普羅米修斯服務時，它給出的錯誤Could not resolve host: prometheus-server;

我開始故障排除並執行以下步驟

• 驗證 podsCIDR 已配置

  kubectl get nodes -o jsonpath='{.items[*].spec.podCIDR}' 10.244.0.0/24

• IP 上的 Curl 和 DNS 服務名稱

  # curl 10.244.0.33:9090 <a href="/prometheus/graph">Found</a>

  # curl 10.109.215.27:9090 <a href="/prometheus/graph">Found</a>

  # curl http://prometheus-server:9090 curl: (6) Could not resolve host: prometheus-server; Unknown error

• 我的 /etc/resolv.conf 是空的，我在下面添加了條目但仍然沒有成功

  search cluster.local nameserver <IP of Server>

• 以下是 CoreDNS 日志的 output

  kubectl logs -f coredns-6955765f44-cnhtz -n kube-system.:53 [INFO] plugin/reload: Running configuration MD5 = 4e235fcc3696966e76816bcd9034ebc7 CoreDNS-1.6.5 linux/amd64, go1.13.4, c2fd1b2 [ERROR] plugin/errors: 2 2339874627451903403.2757028323724952357. HINFO: read udp 10.244.0.13:38879->8.8.4.4:53: read: no route to host [ERROR] plugin/errors: 2 2339874627451903403.2757028323724952357. HINFO: read udp 10.244.0.13:53266->8.8.4.4:53: i/o timeout [ERROR] plugin/errors: 2 2339874627451903403.2757028323724952357. HINFO: read udp 10.244.0.13:37289->8.8.8.8:53: i/o timeout [ERROR] plugin/errors: 2 2339874627451903403.2757028323724952357. HINFO: read udp 10.244.0.13:44281->8.8.4.4:53: read: no route to host

更新1：

為了響應@KoopaKiller，我運行 curl http://prometheus-server:9090 ？ 來自主機和 grafana pod（來自 grafana pod 實際上它沒有響應 IP）。 我用清單安裝了 prometheus 和 grafana，它們都在同一個命名空間中。

kubectl get pods -A
NAMESPACE              NAME                                             READY   STATUS          
kube-system            coredns-6955765f44-cnhtz                         1/1     Running         
kube-system            coredns-6955765f44-d9wrj                         1/1     Running         
kube-system            kube-flannel-ds-amd64-rbsbv                      1/1     Running         
kube-system            kube-proxy-nblnq                                 1/1     Running         
monitoring-logging     grafana-b57ccddf9-p7w2q                          1/1     Running                 
monitoring-logging     prometheus-server-65d7dc7999-frd8k               2/2     Running 

我在 coredns 事件中觀察到的另一件事是缺少文件“/run/flannel/subnet.env”，但文件可用，看起來每次重新啟動時都會重新創建，CoreDNS 發現它很晚。

Events:
  Type     Reason                   Message
  ----     ------                   -------
  Warning  FailedCreatePodSandBox   Failed to create pod sandbox: rpc error: code = Unknown desc = failed to set up sandbox container "d69af6411310ae3c4865a3ddce0667a40092b0dcf55eb5f8ddb642e503dcc0c5" network for pod "coredns-6955765f44-d9wrj": networkPlugin cni failed to set up pod "coredns-6955765f44-d9wrj_kube-system" network: open /run/flannel/subnet.env: no such file or directory
  Warning  FailedCreatePodSandBox   Failed to create pod sandbox: rpc error: code = Unknown desc = failed to set up sandbox container "b6199b3ce4a769c0ccfef6f247763beb1ca0231de52f6309d2b2f122844746ee" network for pod "coredns-6955765f44-d9wrj": networkPlugin cni failed to set up pod "coredns-6955765f44-d9wrj_kube-system" network: open /run/flannel/subnet.env: no such file or directory
  Normal   SandboxChanged           Pod sandbox changed, it will be killed and re-created.
  Warning  FailedCreatePodSandBox   Failed to create pod sandbox: rpc error: code = Unknown desc = failed to set up sandbox container "097dbf97858d8ea3510e8337eb9b0bc8baf966ab51a2a56971e8ae54c5b516a6" network for pod "coredns-6955765f44-d9wrj": networkPlugin cni failed to set up pod "coredns-6955765f44-d9wrj_kube-system" network: open /run/flannel/subnet.env: no such file or directory
  Normal   Pulled                   Container image "k8s.gcr.io/coredns:1.6.5" already present on machine
  Normal   Created                  Created container coredns
  Normal   Started                  Started container coredns

Update2：我按照鏈接調試 DNS 並顯示結果

kubectl exec -ti dnsutils -- nslookup kubernetes.default
kubectl exec dnsutils cat /etc/resolv.conf

然后我將日志插件添加到CoreDNS配置中，並實現CoreDNS沒有收到DNS查詢，我禁用了我的firewalld，一切都開始按預期工作，但是為什么它不能與firewalld一起工作，我的開放端口如下，它也有法蘭絨端口

firewall-cmd --list-ports
6443/tcp 2379-2380/tcp 10250/tcp 10251/tcp 10252/tcp 30000-32767/tcp 8080/tcp 8443/tcp 8285/udp 8472/udp 502/tcp

Answer 1

為了使其在不禁用防火牆的情況下工作，我必須添加以下規則，一切都開始使用 dnsnames

firewall-cmd --add-masquerade --permanent
firewall-cmd --reload
systemctl restart firewalld

我從下面的鏈接中得到了提示，但會更詳細地了解我們為什么需要它？

如何在不禁用 firewalld (Kubernetes) 的情況下使用 Flannel