CoreDNS 不使用 K8S/Minikube 解析命名空間外的服務 url

Question

我有一個運行 minikube 1.6.2 的本地集群。

我所有的 pod 都正常，我單獨檢查了日志，但我有 2 db、influx 和 postgres，不能再從命名空間外的任何 url 訪問。

我登錄了兩個 pod，我可以確認每個 db 都正常，有數據，我可以手動連接我的用戶/密碼。

讓我們以涌入的情況為例。

kubectl exec -it -n influx blockchain-influxdb-local-fb745b98c-vbghp -- influx -username='myuser' -password="mypass" -database="mydb" -precision=rfc3339 -execute "show measurements"

給了我 4 個測量值，所以沒有 pb。

但是當我嘗試將來自同一命名空間的涌入與他的本地 dns 連接時，我得到了超時。

➜ ~ kubectl get svc -n influx 
NAME                        TYPE        CLUSTER-IP     EXTERNAL-IP   PORT(S)     AGE
blockchain-influxdb-local   ClusterIP   10.96.175.62   <none>        8086/TCP    19m


➜ ~ kubectl get deployments -n influx
NAME                        READY   UP-TO-DATE   AVAILABLE   AGE
blockchain-influxdb-local   1/1     1            1           20m


➜  ~ kubectl get po -n influx
NAME                                        READY   STATUS    RESTARTS   AGE
blockchain-influxdb-local-fb745b98c-vbghp   1/1     Running   0          21m
measures-api-local-8667bb496f-4wp8d         1/1     Running   0          21m

工作情況：

從同一命名空間內的 pod：

curl --verbose -G 'http://blockchain-influxdb-local:8086/query?db=mydb&pretty=true' --data-urlencode 'u=myuser' --data-urlencode 'p=mypass' --data-urlencode 'precision=rfc3339' --data-urlencode 'q=show measurements'

來自另一個命名空間（相同命名空間）中的 pod，具有 pod IP

curl --verbose -G '172.17.0.5:8086/query?db=mydb&pretty=true' --data-urlencode 'u=myuser' --data-urlencode 'p=mypass' --data-urlencode 'precision=rfc3339' --data-urlencode 'q=show measurements'

來自另一個命名空間（相同命名空間）中的 pod，具有服務 IP

curl --verbose -G '10.96.175.62:8086/query?db=mydb&pretty=true' --data-urlencode 'u=myuser' --data-urlencode 'p=mypass' --data-urlencode 'precision=rfc3339' --data-urlencode 'q=show measurements'

但是當我從外部命名空間使用本地 dns 時，它不起作用，我從 CURL 得到超時：

 curl --verbose -G 'blockchain-influxdb-local.influx.svc.cluster.local:8086/query?db=mydb&pretty=true' --data-urlencode 'u=myuser' --data-urlencode 'p=mypass' --data-urlencode 'precision=rfc3339' --data-urlencode 'q=show measurements'

我按照這些調試步驟來確保 DNS 正常工作，沒有問題，一切正常。

https://kubernetes.io/docs/tasks/administer-cluster/dns-debugging-resolution/

在同一個 pod 中，當我 ping 這個 url 時，我得到：

root@metadata-api-local-8b4b7846b-zllb8:/go/src/gitlab.com/company/metadata_api# ping blockchain-influxdb-local.influx.svc.cluster.local

PING nc-ass-vip.sdv.fr (212.95.74.75) 56(84) bytes of data.
--- nc-ass-vip.sdv.fr ping statistics ---
3 packets transmitted, 0 received, 100% packet loss, time 47ms

我不知道為什么要引用nc-ass-vip.sdv.fr

我還嘗試刪除本地集群並重新部署它，還嘗試將 minikube 更新到最新版本（1.8.2），但沒有任何效果。

我不知道還能做什么...

有人有什么想法嗎？ 我工作了好幾個月，真的不知道發生了什么。 :(

為了回應@Arghya Sadhu，我從 Influx pod 中發布了文件 /etc/resolv.conf：

nameserver 10.96.0.10
search influx.svc.cluster.local svc.cluster.local cluster.local numericable.fr
options ndots:5

kubectl edit cm coredns -n kube-system

# Please edit the object below. Lines beginning with a '#' will be ignored,
# and an empty file will abort the edit. If an error occurs while saving this file will be
# reopened with the relevant failures.
#
apiVersion: v1
data:
  Corefile: |
    .:53 {
        errors
        health {
           lameduck 5s
        }
        ready
        kubernetes cluster.local in-addr.arpa ip6.arpa {
           pods insecure
           fallthrough in-addr.arpa ip6.arpa
           ttl 30
        }
        prometheus :9153
        forward . /etc/resolv.conf
        cache 30
        loop
        reload
        loadbalance
    }
kind: ConfigMap
metadata:
  creationTimestamp: "2020-03-19T10:59:28Z"
  name: coredns
  namespace: kube-system
  resourceVersion: "176"
  selfLink: /api/v1/namespaces/kube-system/configmaps/coredns
  uid: 0797c1a9-e9db-4b4c-bc8d-4c7ecca24968

編輯：

kubectl exec -ti dnsutils -- nslookup blockchain-influxdb-local.influx.svc.cluster.local                                         
Server:     10.96.0.10
Address:    10.96.0.10#53

Non-authoritative answer:
blockchain-influxdb-local.influx.svc.cluster.local.numericable.fr   canonical name = nc-ass-vip.sdv.fr.
Name:   nc-ass-vip.sdv.fr
Address: 212.95.74.75

Answer 1

在深入研究了幾種可能性之后，我們發現了以下命令的輸出：

$ kubectl run dnsutils -it --rm=true --restart=Never --image=tutum/dnsutils -- nslookup -debug blockchain-influxdb-local.influx

$ kubectl run dnsutils -it --rm=true --restart=Never --image=tutum/dnsutils -- nslookup -debug blockchain-influxdb-local.influx.svc.cluster.local

$ kubectl run dnsutils -it --rm=true --restart=Never --image=tutum/dnsutils -- nslookup -debug blockchain-influxdb-local.influx.svc.cluster.local.

這些命令的輸出在這里（添加到此答案的末尾以供將來參考，以防鏈接不起作用）。

查看此輸出我們可以看到，無論numericable.fr總是對 dns 查詢給出肯定的回答。

為避免這種情況，您可以將 pod 中的 ndots 條目更改為 1 甚至 0。

nameserver 10.96.0.10
search influx.svc.cluster.local svc.cluster.local cluster.local numericable.fr
options ndots:0

從手冊頁我們有：

ndots:n 設置在進行初始絕對查詢之前必須出現在指定給 res_query(3)（請參閱 resolver(3)）的名稱中的點數的閾值。 n 的默認值是 1，這意味着如果名稱中有任何點，則在將任何搜索列表元素附加到名稱之前，將首先嘗試將該名稱作為絕對名稱。 此選項的值以靜默方式限制為 15。

更有效和長期的解決方案是在 pod/statefulset/deployment 清單中添加此條目，如下例所示：

apiVersion: v1
kind: Pod
metadata:
  namespace: default
  name: dns-example
spec:
  containers:
    - name: test
      image: nginx
  dnsConfig:
    options:
      - name: ndots
        value: "0"

參考命令的輸出以供將來參考：

➜  ~ kubectl run dnsutils -it --rm=true --restart=Never --image=tutum/dnsutils -- nslookup -debug blockchain-influxdb-local.influx 

Server:     10.96.0.10
Address:    10.96.0.10#53

------------
    QUESTIONS:
    blockchain-influxdb-local.influx.default.svc.cluster.local, type = A, class = IN
    ANSWERS:
    AUTHORITY RECORDS:
    ->  cluster.local
    origin = ns.dns.cluster.local
    mail addr = hostmaster.cluster.local
    serial = 1584628757
    refresh = 7200
    retry = 1800
    expire = 86400
    minimum = 30
    ttl = 10
    ADDITIONAL RECORDS:
------------
** server can't find blockchain-influxdb-local.influx.default.svc.cluster.local: NXDOMAIN
Server:     10.96.0.10
Address:    10.96.0.10#53

------------
    QUESTIONS:
    blockchain-influxdb-local.influx.svc.cluster.local, type = A, class = IN
    ANSWERS:
    ->  blockchain-influxdb-local.influx.svc.cluster.local
    internet address = 10.96.72.6
    ttl = 10
    AUTHORITY RECORDS:
    ADDITIONAL RECORDS:
------------
Name:   blockchain-influxdb-local.influx.svc.cluster.local
Address: 10.96.72.6

pod "dnsutils" deleted
pod default/dnsutils terminated (Error)
➜  ~ kubectl run dnsutils -it --rm=true --restart=Never --image=tutum/dnsutils -- nslookup -debug blockchain-influxdb-local.influx.svc.cluster.local 

Server:     10.96.0.10
Address:    10.96.0.10#53

------------
    QUESTIONS:
    blockchain-influxdb-local.influx.svc.cluster.local.default.svc.cluster.local, type = A, class = IN
    ANSWERS:
    AUTHORITY RECORDS:
    ->  cluster.local
    origin = ns.dns.cluster.local
    mail addr = hostmaster.cluster.local
    serial = 1584628757
    refresh = 7200
    retry = 1800
    expire = 86400
    minimum = 30
    ttl = 30
    ADDITIONAL RECORDS:
------------
** server can't find blockchain-influxdb-local.influx.svc.cluster.local.default.svc.cluster.local: NXDOMAIN
Server:     10.96.0.10
Address:    10.96.0.10#53

------------
    QUESTIONS:
    blockchain-influxdb-local.influx.svc.cluster.local.svc.cluster.local, type = A, class = IN
    ANSWERS:
    AUTHORITY RECORDS:
    ->  cluster.local
    origin = ns.dns.cluster.local
    mail addr = hostmaster.cluster.local
    serial = 1584628757
    refresh = 7200
    retry = 1800
    expire = 86400
    minimum = 30
    ttl = 30
    ADDITIONAL RECORDS:
------------
** server can't find blockchain-influxdb-local.influx.svc.cluster.local.svc.cluster.local: NXDOMAIN
Server:     10.96.0.10
Address:    10.96.0.10#53

------------
    QUESTIONS:
    blockchain-influxdb-local.influx.svc.cluster.local.cluster.local, type = A, class = IN
    ANSWERS:
    AUTHORITY RECORDS:
    ->  cluster.local
    origin = ns.dns.cluster.local
    mail addr = hostmaster.cluster.local
    serial = 1584628757
    refresh = 7200
    retry = 1800
    expire = 86400
    minimum = 30
    ttl = 30
    ADDITIONAL RECORDS:
------------
** server can't find blockchain-influxdb-local.influx.svc.cluster.local.cluster.local: NXDOMAIN
Server:     10.96.0.10
Address:    10.96.0.10#53

------------
    QUESTIONS:
    blockchain-influxdb-local.influx.svc.cluster.local.numericable.fr, type = A, class = IN
    ANSWERS:
    ->  blockchain-influxdb-local.influx.svc.cluster.local.numericable.fr
    canonical name = nc-ass-vip.sdv.fr.
    ttl = 30
    ->  nc-ass-vip.sdv.fr
    internet address = 212.95.74.75
    ttl = 30
    AUTHORITY RECORDS:
    ADDITIONAL RECORDS:
------------
Non-authoritative answer:
blockchain-influxdb-local.influx.svc.cluster.local.numericable.fr   canonical name = nc-ass-vip.sdv.fr.
Name:   nc-ass-vip.sdv.fr
Address: 212.95.74.75

pod "dnsutils" deleted
pod default/dnsutils terminated (Error)
➜  ~ kubectl run dnsutils -it --rm=true --restart=Never --image=tutum/dnsutils -- nslookup -debug blockchain-influxdb-local.influx.svc.cluster.local.
Server:     10.96.0.10
Address:    10.96.0.10#53

------------
    QUESTIONS:
    blockchain-influxdb-local.influx.svc.cluster.local, type = A, class = IN
    ANSWERS:
    ->  blockchain-influxdb-local.influx.svc.cluster.local
    internet address = 10.96.72.6
    ttl = 30
    AUTHORITY RECORDS:
    ADDITIONAL RECORDS:
------------
Name:   blockchain-influxdb-local.influx.svc.cluster.local
Address: 10.96.72.6

pod "dnsutils" deleted