CoreDNS 不使用 K8S/Minikube 解析命名空间外的服务 url

Question

我有一个运行 minikube 1.6.2 的本地集群。

我所有的 pod 都正常，我单独检查了日志，但我有 2 db、influx 和 postgres，不能再从命名空间外的任何 url 访问。

我登录了两个 pod，我可以确认每个 db 都正常，有数据，我可以手动连接我的用户/密码。

让我们以涌入的情况为例。

kubectl exec -it -n influx blockchain-influxdb-local-fb745b98c-vbghp -- influx -username='myuser' -password="mypass" -database="mydb" -precision=rfc3339 -execute "show measurements"

给了我 4 个测量值，所以没有 pb。

但是当我尝试将来自同一命名空间的涌入与他的本地 dns 连接时，我得到了超时。

➜ ~ kubectl get svc -n influx 
NAME                        TYPE        CLUSTER-IP     EXTERNAL-IP   PORT(S)     AGE
blockchain-influxdb-local   ClusterIP   10.96.175.62   <none>        8086/TCP    19m


➜ ~ kubectl get deployments -n influx
NAME                        READY   UP-TO-DATE   AVAILABLE   AGE
blockchain-influxdb-local   1/1     1            1           20m


➜  ~ kubectl get po -n influx
NAME                                        READY   STATUS    RESTARTS   AGE
blockchain-influxdb-local-fb745b98c-vbghp   1/1     Running   0          21m
measures-api-local-8667bb496f-4wp8d         1/1     Running   0          21m

工作情况：

从同一命名空间内的 pod：

curl --verbose -G 'http://blockchain-influxdb-local:8086/query?db=mydb&pretty=true' --data-urlencode 'u=myuser' --data-urlencode 'p=mypass' --data-urlencode 'precision=rfc3339' --data-urlencode 'q=show measurements'

来自另一个命名空间（相同命名空间）中的 pod，具有 pod IP

curl --verbose -G '172.17.0.5:8086/query?db=mydb&pretty=true' --data-urlencode 'u=myuser' --data-urlencode 'p=mypass' --data-urlencode 'precision=rfc3339' --data-urlencode 'q=show measurements'

来自另一个命名空间（相同命名空间）中的 pod，具有服务 IP

curl --verbose -G '10.96.175.62:8086/query?db=mydb&pretty=true' --data-urlencode 'u=myuser' --data-urlencode 'p=mypass' --data-urlencode 'precision=rfc3339' --data-urlencode 'q=show measurements'

但是当我从外部命名空间使用本地 dns 时，它不起作用，我从 CURL 得到超时：

 curl --verbose -G 'blockchain-influxdb-local.influx.svc.cluster.local:8086/query?db=mydb&pretty=true' --data-urlencode 'u=myuser' --data-urlencode 'p=mypass' --data-urlencode 'precision=rfc3339' --data-urlencode 'q=show measurements'

我按照这些调试步骤来确保 DNS 正常工作，没有问题，一切正常。

https://kubernetes.io/docs/tasks/administer-cluster/dns-debugging-resolution/

在同一个 pod 中，当我 ping 这个 url 时，我得到：

root@metadata-api-local-8b4b7846b-zllb8:/go/src/gitlab.com/company/metadata_api# ping blockchain-influxdb-local.influx.svc.cluster.local

PING nc-ass-vip.sdv.fr (212.95.74.75) 56(84) bytes of data.
--- nc-ass-vip.sdv.fr ping statistics ---
3 packets transmitted, 0 received, 100% packet loss, time 47ms

我不知道为什么要引用nc-ass-vip.sdv.fr

我还尝试删除本地集群并重新部署它，还尝试将 minikube 更新到最新版本（1.8.2），但没有任何效果。

我不知道还能做什么...

有人有什么想法吗？ 我工作了好几个月，真的不知道发生了什么。 :(

为了回应@Arghya Sadhu，我从 Influx pod 中发布了文件 /etc/resolv.conf：

nameserver 10.96.0.10
search influx.svc.cluster.local svc.cluster.local cluster.local numericable.fr
options ndots:5

kubectl edit cm coredns -n kube-system

# Please edit the object below. Lines beginning with a '#' will be ignored,
# and an empty file will abort the edit. If an error occurs while saving this file will be
# reopened with the relevant failures.
#
apiVersion: v1
data:
  Corefile: |
    .:53 {
        errors
        health {
           lameduck 5s
        }
        ready
        kubernetes cluster.local in-addr.arpa ip6.arpa {
           pods insecure
           fallthrough in-addr.arpa ip6.arpa
           ttl 30
        }
        prometheus :9153
        forward . /etc/resolv.conf
        cache 30
        loop
        reload
        loadbalance
    }
kind: ConfigMap
metadata:
  creationTimestamp: "2020-03-19T10:59:28Z"
  name: coredns
  namespace: kube-system
  resourceVersion: "176"
  selfLink: /api/v1/namespaces/kube-system/configmaps/coredns
  uid: 0797c1a9-e9db-4b4c-bc8d-4c7ecca24968

编辑：

kubectl exec -ti dnsutils -- nslookup blockchain-influxdb-local.influx.svc.cluster.local                                         
Server:     10.96.0.10
Address:    10.96.0.10#53

Non-authoritative answer:
blockchain-influxdb-local.influx.svc.cluster.local.numericable.fr   canonical name = nc-ass-vip.sdv.fr.
Name:   nc-ass-vip.sdv.fr
Address: 212.95.74.75

Answer 1

在深入研究了几种可能性之后，我们发现了以下命令的输出：

$ kubectl run dnsutils -it --rm=true --restart=Never --image=tutum/dnsutils -- nslookup -debug blockchain-influxdb-local.influx

$ kubectl run dnsutils -it --rm=true --restart=Never --image=tutum/dnsutils -- nslookup -debug blockchain-influxdb-local.influx.svc.cluster.local

$ kubectl run dnsutils -it --rm=true --restart=Never --image=tutum/dnsutils -- nslookup -debug blockchain-influxdb-local.influx.svc.cluster.local.

这些命令的输出在这里（添加到此答案的末尾以供将来参考，以防链接不起作用）。

查看此输出我们可以看到，无论numericable.fr总是对 dns 查询给出肯定的回答。

为避免这种情况，您可以将 pod 中的 ndots 条目更改为 1 甚至 0。

nameserver 10.96.0.10
search influx.svc.cluster.local svc.cluster.local cluster.local numericable.fr
options ndots:0

从手册页我们有：

ndots:n 设置在进行初始绝对查询之前必须出现在指定给 res_query(3)（请参阅 resolver(3)）的名称中的点数的阈值。 n 的默认值是 1，这意味着如果名称中有任何点，则在将任何搜索列表元素附加到名称之前，将首先尝试将该名称作为绝对名称。 此选项的值以静默方式限制为 15。

更有效和长期的解决方案是在 pod/statefulset/deployment 清单中添加此条目，如下例所示：

apiVersion: v1
kind: Pod
metadata:
  namespace: default
  name: dns-example
spec:
  containers:
    - name: test
      image: nginx
  dnsConfig:
    options:
      - name: ndots
        value: "0"

参考命令的输出以供将来参考：

➜  ~ kubectl run dnsutils -it --rm=true --restart=Never --image=tutum/dnsutils -- nslookup -debug blockchain-influxdb-local.influx 

Server:     10.96.0.10
Address:    10.96.0.10#53

------------
    QUESTIONS:
    blockchain-influxdb-local.influx.default.svc.cluster.local, type = A, class = IN
    ANSWERS:
    AUTHORITY RECORDS:
    ->  cluster.local
    origin = ns.dns.cluster.local
    mail addr = hostmaster.cluster.local
    serial = 1584628757
    refresh = 7200
    retry = 1800
    expire = 86400
    minimum = 30
    ttl = 10
    ADDITIONAL RECORDS:
------------
** server can't find blockchain-influxdb-local.influx.default.svc.cluster.local: NXDOMAIN
Server:     10.96.0.10
Address:    10.96.0.10#53

------------
    QUESTIONS:
    blockchain-influxdb-local.influx.svc.cluster.local, type = A, class = IN
    ANSWERS:
    ->  blockchain-influxdb-local.influx.svc.cluster.local
    internet address = 10.96.72.6
    ttl = 10
    AUTHORITY RECORDS:
    ADDITIONAL RECORDS:
------------
Name:   blockchain-influxdb-local.influx.svc.cluster.local
Address: 10.96.72.6

pod "dnsutils" deleted
pod default/dnsutils terminated (Error)
➜  ~ kubectl run dnsutils -it --rm=true --restart=Never --image=tutum/dnsutils -- nslookup -debug blockchain-influxdb-local.influx.svc.cluster.local 

Server:     10.96.0.10
Address:    10.96.0.10#53

------------
    QUESTIONS:
    blockchain-influxdb-local.influx.svc.cluster.local.default.svc.cluster.local, type = A, class = IN
    ANSWERS:
    AUTHORITY RECORDS:
    ->  cluster.local
    origin = ns.dns.cluster.local
    mail addr = hostmaster.cluster.local
    serial = 1584628757
    refresh = 7200
    retry = 1800
    expire = 86400
    minimum = 30
    ttl = 30
    ADDITIONAL RECORDS:
------------
** server can't find blockchain-influxdb-local.influx.svc.cluster.local.default.svc.cluster.local: NXDOMAIN
Server:     10.96.0.10
Address:    10.96.0.10#53

------------
    QUESTIONS:
    blockchain-influxdb-local.influx.svc.cluster.local.svc.cluster.local, type = A, class = IN
    ANSWERS:
    AUTHORITY RECORDS:
    ->  cluster.local
    origin = ns.dns.cluster.local
    mail addr = hostmaster.cluster.local
    serial = 1584628757
    refresh = 7200
    retry = 1800
    expire = 86400
    minimum = 30
    ttl = 30
    ADDITIONAL RECORDS:
------------
** server can't find blockchain-influxdb-local.influx.svc.cluster.local.svc.cluster.local: NXDOMAIN
Server:     10.96.0.10
Address:    10.96.0.10#53

------------
    QUESTIONS:
    blockchain-influxdb-local.influx.svc.cluster.local.cluster.local, type = A, class = IN
    ANSWERS:
    AUTHORITY RECORDS:
    ->  cluster.local
    origin = ns.dns.cluster.local
    mail addr = hostmaster.cluster.local
    serial = 1584628757
    refresh = 7200
    retry = 1800
    expire = 86400
    minimum = 30
    ttl = 30
    ADDITIONAL RECORDS:
------------
** server can't find blockchain-influxdb-local.influx.svc.cluster.local.cluster.local: NXDOMAIN
Server:     10.96.0.10
Address:    10.96.0.10#53

------------
    QUESTIONS:
    blockchain-influxdb-local.influx.svc.cluster.local.numericable.fr, type = A, class = IN
    ANSWERS:
    ->  blockchain-influxdb-local.influx.svc.cluster.local.numericable.fr
    canonical name = nc-ass-vip.sdv.fr.
    ttl = 30
    ->  nc-ass-vip.sdv.fr
    internet address = 212.95.74.75
    ttl = 30
    AUTHORITY RECORDS:
    ADDITIONAL RECORDS:
------------
Non-authoritative answer:
blockchain-influxdb-local.influx.svc.cluster.local.numericable.fr   canonical name = nc-ass-vip.sdv.fr.
Name:   nc-ass-vip.sdv.fr
Address: 212.95.74.75

pod "dnsutils" deleted
pod default/dnsutils terminated (Error)
➜  ~ kubectl run dnsutils -it --rm=true --restart=Never --image=tutum/dnsutils -- nslookup -debug blockchain-influxdb-local.influx.svc.cluster.local.
Server:     10.96.0.10
Address:    10.96.0.10#53

------------
    QUESTIONS:
    blockchain-influxdb-local.influx.svc.cluster.local, type = A, class = IN
    ANSWERS:
    ->  blockchain-influxdb-local.influx.svc.cluster.local
    internet address = 10.96.72.6
    ttl = 30
    AUTHORITY RECORDS:
    ADDITIONAL RECORDS:
------------
Name:   blockchain-influxdb-local.influx.svc.cluster.local
Address: 10.96.72.6

pod "dnsutils" deleted