Kubernetes服務通訊isse-Kubedns

Question

我有兩個Pod映射到兩個服務，並在筆記本電腦中使用虛擬盒虛擬機運行。 我有kube dns工作。 一個pod是Web服務，另一個是mongodb。

下面是webapp pod的規格

spec:
  containers:
    - resources:
        limits:
          cpu: 0.5
          .
          .
      name: wsemp
      ports:
        - containerPort: 8080
  #     name: wsemp
  #command: ["java","-Dspring.data.mongodb.uri=mongodb://192.168.6.103:30061/microservices", "-Djava.security.egd=file:/dev/./urandom","-jar","/app.jar"]
  command: ["java","-Dspring.data.mongodb.uri=mongodb://mongoservice/microservices", "-Djava.security.egd=file:/dev/./urandom","-jar","/app.jar"]

相應服務的規格

apiVersion: v1
kind: Service
metadata:
  labels:
    name: webappservice
  name: webappservice
spec:
  ports:
   - port: 8080
     nodePort: 30062
     targetPort: 8080
     protocol: TCP
  type: NodePort
  selector:
    name: webapp

Mongodb Pod規格

apiVersion: v1
kind: Pod
metadata:
  name: mongodb
  labels:
    name: mongodb
spec:
  containers:
    .
    .
  name: mongodb
  ports:
    - containerPort: 27017

Mongodb服務規格

apiVersion: v1
kind: Service
metadata:
  labels:
    name: mongodb
  name: mongoservice
spec:
  ports:
   - port: 27017
     nodePort: 30061
     targetPort: 27017
     protocol: TCP
  type: NodePort
  selector:
    name: mongodb

評論后服務中更新的目標端口

問題

webapp啟動時無法與mongoservice端口連接，並在啟動時給出此錯誤

Exception in monitor thread while connecting to server mongoservice:27017
com.mongodb.MongoSocketOpenException: Exception opening socket
at com.mongodb.connection.SocketStream.open(SocketStream.java:63) ~[mongodb-driver-core-3.2.2.jar!/:na]
at        com.mongodb.connection.InternalStreamConnection.open(InternalStreamConnection.java:114) ~[mongodb-driver-core-3.2.2.jar!/:na]
at com.mongodb.connection.DefaultServerMonitor$ServerMonitorRunnable.run(DefaultServerMonitor.java:128) ~[mongodb-driver-core-3.2.2.jar!/:na]
at java.lang.Thread.run(Thread.java:745) [na:1.8.0_111]
Caused by: java.net.ConnectException: Connection refused (Connection refused)
at java.net.PlainSocketImpl.socketConnect(Native Method) ~[na:1.8.0_111]

描述svc

kubectl describe svc mongoservice
Name:           mongoservice
Namespace:      default
Labels:         name=mongodb
Selector:       name=mongodb
Type:           NodePort
IP:         10.254.146.189
Port:           <unset> 27017/TCP
NodePort:       <unset> 30061/TCP
Endpoints:      172.17.99.2:27017
Session Affinity:   None
No events.

kubectl describe svc webappservice 
Name:           webappservice
Namespace:      default
Labels:         name=webappservice
Selector:       name=webapp
Type:           NodePort
IP:         10.254.112.121
Port:           <unset> 8080/TCP
NodePort:       <unset> 30062/TCP
Endpoints:      172.17.99.3:8080
Session Affinity:   None
No events.

調試

root@webapp:/# nslookup mongoservice
Server:     10.254.0.2
Address:    10.254.0.2#53

Non-authoritative answer:
Name:   mongoservice.default.svc.cluster.local
Address: 10.254.146.189

root@webapp:/# curl 10.254.146.189:27017
curl: (7) Failed to connect to 10.254.146.189 port 27017: Connection refused
root@webapp:/# curl mongoservice:27017
curl: (7) Failed to connect to mongoservice port 27017: Connection refused


sudo iptables-save | grep webapp

-A KUBE-NODEPORTS -p tcp -m comment --comment "default/webappservice:" -m tcp --dport 30062 -j KUBE-MARK-MASQ
-A KUBE-NODEPORTS -p tcp -m comment --comment "default/webappservice:" -m tcp --dport 30062 -j KUBE-SVC-NQBDRRKQULANV7O3
-A KUBE-SEP-IE7EBTQCN7T6HXC4 -s 172.17.99.3/32 -m comment --comment "default/webappservice:" -j KUBE-MARK-MASQ
-A KUBE-SEP-IE7EBTQCN7T6HXC4 -p tcp -m comment --comment "default/webappservice:" -m tcp -j DNAT --to-destination 172.17.99.3:8080
-A KUBE-SERVICES -d 10.254.217.24/32 -p tcp -m comment --comment "default/webappservice: cluster IP" -m tcp --dport 8080 -j KUBE-SVC-NQBDRRKQULANV7O3
-A KUBE-SVC-NQBDRRKQULANV7O3 -m comment --comment "default/webappservice:" -j KUBE-SEP-IE7EBTQCN7T6HXC4
$ curl 10.254.217.24:8080
{"timestamp":1486678423757,"status":404,"error":"Not Found","message":"No message available","path":"/"}[osboxes@kube-node1 ~]$ 


sudo iptables-save | grep mongodb
[osboxes@osboxes ~]$ sudo iptables-save | grep mongo
-A KUBE-NODEPORTS -p tcp -m comment --comment "default/mongoservice:" -m tcp --dport 30061 -j KUBE-MARK-MASQ
-A KUBE-NODEPORTS -p tcp -m comment --comment "default/mongoservice:" -m tcp --dport 30061 -j KUBE-SVC-2HQWGC3WSIBZF7CN
-A KUBE-SEP-FVWOWAWXXVAVIQ5O -s 172.17.99.2/32 -m comment --comment "default/mongoservice:" -j KUBE-MARK-MASQ
-A KUBE-SEP-FVWOWAWXXVAVIQ5O -p tcp -m comment --comment "default/mongoservice:" -m tcp -j DNAT --to-destination 172.17.99.2:27017
-A KUBE-SERVICES -d 10.254.146.189/32 -p tcp -m comment --comment "default/mongoservice: cluster IP" -m tcp --dport 27017 -j KUBE-SVC-2HQWGC3WSIBZF7CN
-A KUBE-SVC-2HQWGC3WSIBZF7CN -m comment --comment "default/mongoservice:" -j KUBE-SEP-FVWOWAWXXVAVIQ5O
[osboxes@osboxes ~]$ sudo curl  10.254.146.189:8080
^C[osboxes@osboxes ~]$ sudo curl  10.254.146.189:27017

It looks like you are trying to access MongoDB over HTTP on the native driver port.


root@mongodb:/# netstat -an
Active Internet connections (servers and established)
Proto Recv-Q Send-Q Local Address           Foreign Address         State      
tcp        0      0 0.0.0.0:27017           0.0.0.0:*               LISTEN     
tcp        0      0 172.17.99.2:60724       151.101.128.204:80      TIME_WAIT  
tcp        0      0 172.17.99.2:60728       151.101.128.204:80      TIME_WAIT  

mongodb容器在啟動時沒有錯誤。

嘗試遵循https://kubernetes.io/docs/user-guide/debugging-services/#iptables中的步驟進行操作，因為我說“嘗試將-V標志設置為4重啟kube-proxy”不知道該怎么做。

我不是網絡專家，所以不知道該如何進行分析。 任何調試技巧都將提供很大的幫助。

謝謝。

Answer 1

:)

附帶說明一下，請記住，curl默認情況下會執行HTTP請求，但是您嘗試訪問的主機中的端口27017未綁定到能夠理解此類協議的應用程序。 通常，在這些情況下，您將要使用netcat：

nc -zv mongoservice 27017

這將報告來自該主機的端口27017是否打開。

• nc = netcat
• -z掃描偵聽守護程序而不發送數據
• -v增加詳細程度

關於您的MongoDB文件，您必須記住要設置targetPort指令。 如Kubernetes文檔中有關targetPort的解釋：

This specification will create a Service which targets TCP port 80 on any Pod with the run: my-nginx label, and expose it on an abstracted Service port (targetPort: is the port the container accepts traffic on, port: is the abstracted Service port, which can be any port other pods use to access the Service). View service API object to see the list of supported fields in service definition.

因此，只需將其設置為27017即可保持一致性。

遵循這些建議后，您不應遇到任何問題。 保持良好的工作，並盡可能多地學習！

Answer 2

iptables規則看起來還可以，但是不確定您的kubernetes中使用了哪種網絡解決方案（法蘭絨/印花棉布）。 您可以檢查是否可以從Web Pod訪問kube dns Pod IP。

Answer 3

謝謝。 我對此有所了解，並且由於我正在使用絨布網絡，因此絨布網絡中的Pod之間的通信存在問題。

特別是這部分，如鏈接http://jayunit100.blogspot.com/2015/06/flannel-and-vagrant-heads-up.html中提到的FLANNEL_OPTIONS =“-iface = eth1”

謝謝。