docker swarm on oracle 雲實例導致覆蓋網絡不可達
[英]docker swarm on oracle cloud instance results in unreachable overlay network
問題描述
我的 docker 群設置由 3 個節點組成
- 家庭服務器
anton - 數字海洋
zeus - oracle 雲永遠免費 ARM 實例
galio
我正在使用traefik/whoami測試連接性，將其部署在所有這 3 個節點上，並在zeus上設置了traefik負載均衡器。
發出請求時，它僅適用於 2 個第一個請求，但第三個請求失敗，並顯示 502 或 504 HTTP 狀態碼。 以及 traefik 報告caused by: dial tcp 10.0.1.27:80: connect: no route to host" ，其中指定地址為 galio 的galio
Oracle 雲子網安全列表
IPTables 配置
# Generated by iptables-save v1.8.7 on Sun Sep 4 23:12:52 2022
*filter
:INPUT ACCEPT [9:1455]
:FORWARD ACCEPT [0:0]
:OUTPUT ACCEPT [12503:4589221]
:DOCKER - [0:0]
:DOCKER-ISOLATION-STAGE-1 - [0:0]
:DOCKER-ISOLATION-STAGE-2 - [0:0]
:DOCKER-USER - [0:0]
:InstanceServices - [0:0]
-A INPUT -m state --state RELATED,ESTABLISHED -j ACCEPT
-A INPUT -p icmp -j ACCEPT
-A INPUT -i lo -j ACCEPT
-A INPUT -p udp -m udp --sport 123 -j ACCEPT
-A INPUT -p tcp -m state --state NEW -m tcp --dport 22 -j ACCEPT
-A INPUT -p tcp -m state --state NEW -m tcp --dport 2376 -j ACCEPT
-A INPUT -p tcp -m state --state NEW -m tcp --dport 7946 -j ACCEPT
-A INPUT -p udp -m state --state NEW -m udp --dport 7946 -j ACCEPT
-A INPUT -p udp -m state --state NEW -m udp --dport 4789 -j ACCEPT
-A FORWARD -j DOCKER-USER
-A FORWARD -j DOCKER-ISOLATION-STAGE-1
-A FORWARD -o docker0 -m conntrack --ctstate RELATED,ESTABLISHED -j ACCEPT
-A FORWARD -o docker0 -j DOCKER
-A FORWARD -i docker0 ! -o docker0 -j ACCEPT
-A FORWARD -i docker0 -o docker0 -j ACCEPT
-A FORWARD -o docker_gwbridge -m conntrack --ctstate RELATED,ESTABLISHED -j ACCEPT
-A FORWARD -o docker_gwbridge -j DOCKER
-A FORWARD -i docker_gwbridge ! -o docker_gwbridge -j ACCEPT
-A OUTPUT -d 169.254.0.0/16 -j InstanceServices
-A DOCKER -d 172.18.0.4/32 ! -i docker_gwbridge -o docker_gwbridge -p tcp -m tcp --dport 80 -j ACCEPT
-A DOCKER-ISOLATION-STAGE-1 -i docker0 ! -o docker0 -j DOCKER-ISOLATION-STAGE-2
-A DOCKER-ISOLATION-STAGE-1 -i docker_gwbridge ! -o docker_gwbridge -j DOCKER-ISOLATION-STAGE-2
-A DOCKER-ISOLATION-STAGE-1 -j RETURN
-A DOCKER-ISOLATION-STAGE-2 -j RETURN
-A DOCKER-USER -j RETURN
-A InstanceServices -d 169.254.0.2/32 -p tcp -m owner --uid-owner 0 -m tcp --dport 3260 -m comment --comment "See the Oracle-Provided Images section in the Oracle Cloud Infrastructure documentation for security impact of modifying or removing this rule" -j ACCEPT
-A InstanceServices -d 169.254.2.0/24 -p tcp -m owner --uid-owner 0 -m tcp --dport 3260 -m comment --comment "See the Oracle-Provided Images section in the Oracle Cloud Infrastructure documentation for security impact of modifying or removing this rule" -j ACCEPT
-A InstanceServices -d 169.254.4.0/24 -p tcp -m owner --uid-owner 0 -m tcp --dport 3260 -m comment --comment "See the Oracle-Provided Images section in the Oracle Cloud Infrastructure documentation for security impact of modifying or removing this rule" -j ACCEPT
-A InstanceServices -d 169.254.5.0/24 -p tcp -m owner --uid-owner 0 -m tcp --dport 3260 -m comment --comment "See the Oracle-Provided Images section in the Oracle Cloud Infrastructure documentation for security impact of modifying or removing this rule" -j ACCEPT
-A InstanceServices -d 169.254.0.2/32 -p tcp -m tcp --dport 80 -m comment --comment "See the Oracle-Provided Images section in the Oracle Cloud Infrastructure documentation for security impact of modifying or removing this rule" -j ACCEPT
-A InstanceServices -d 169.254.169.254/32 -p udp -m udp --dport 53 -m comment --comment "See the Oracle-Provided Images section in the Oracle Cloud Infrastructure documentation for security impact of modifying or removing this rule" -j ACCEPT
-A InstanceServices -d 169.254.169.254/32 -p tcp -m tcp --dport 53 -m comment --comment "See the Oracle-Provided Images section in the Oracle Cloud Infrastructure documentation for security impact of modifying or removing this rule" -j ACCEPT
-A InstanceServices -d 169.254.0.3/32 -p tcp -m owner --uid-owner 0 -m tcp --dport 80 -m comment --comment "See the Oracle-Provided Images section in the Oracle Cloud Infrastructure documentation for security impact of modifying or removing this rule" -j ACCEPT
-A InstanceServices -d 169.254.0.4/32 -p tcp -m tcp --dport 80 -m comment --comment "See the Oracle-Provided Images section in the Oracle Cloud Infrastructure documentation for security impact of modifying or removing this rule" -j ACCEPT
-A InstanceServices -d 169.254.169.254/32 -p tcp -m tcp --dport 80 -m comment --comment "See the Oracle-Provided Images section in the Oracle Cloud Infrastructure documentation for security impact of modifying or removing this rule" -j ACCEPT
-A InstanceServices -d 169.254.169.254/32 -p udp -m udp --dport 67 -m comment --comment "See the Oracle-Provided Images section in the Oracle Cloud Infrastructure documentation for security impact of modifying or removing this rule" -j ACCEPT
-A InstanceServices -d 169.254.169.254/32 -p udp -m udp --dport 69 -m comment --comment "See the Oracle-Provided Images section in the Oracle Cloud Infrastructure documentation for security impact of modifying or removing this rule" -j ACCEPT
-A InstanceServices -d 169.254.169.254/32 -p udp -m udp --dport 123 -m comment --comment "See the Oracle-Provided Images section in the Oracle Cloud Infrastructure documentation for security impact of modifying or removing this rule" -j ACCEPT
COMMIT
# Completed on Sun Sep 4 23:12:52 2022
# Generated by iptables-save v1.8.7 on Sun Sep 4 23:12:52 2022
*nat
:PREROUTING ACCEPT [0:0]
:INPUT ACCEPT [0:0]
:OUTPUT ACCEPT [0:0]
:POSTROUTING ACCEPT [0:0]
:DOCKER - [0:0]
-A PREROUTING -m addrtype --dst-type LOCAL -j DOCKER
-A OUTPUT ! -d 127.0.0.0/8 -m addrtype --dst-type LOCAL -j DOCKER
-A POSTROUTING -s 172.17.0.0/16 ! -o docker0 -j MASQUERADE
-A POSTROUTING -s 172.18.0.0/16 ! -o docker_gwbridge -j MASQUERADE
-A POSTROUTING -s 172.18.0.4/32 -d 172.18.0.4/32 -p tcp -m tcp --dport 80 -j MASQUERADE
-A DOCKER -i docker0 -j RETURN
-A DOCKER -i docker_gwbridge -j RETURN
-A DOCKER ! -i docker_gwbridge -p tcp -m tcp --dport 1234 -j DNAT --to-destination 172.18.0.4:80
COMMIT
# Completed on Sun Sep 4 23:12:52 2022
Docker 網絡
gbaranski@zeus > docker network inspect trafik-public
[
{
"Name": "traefik-public",
"Id": "xkpqua30neu7abr9ldw86klfr",
"Created": "2022-09-02T04:38:24.799603491Z",
"Scope": "swarm",
"Driver": "overlay",
"EnableIPv6": false,
"IPAM": {
"Driver": "default",
"Options": null,
"Config": [
{
"Subnet": "10.0.1.0/24",
"Gateway": "10.0.1.1"
}
]
},
"Internal": false,
"Attachable": false,
"Ingress": false,
"ConfigFrom": {
"Network": ""
},
"ConfigOnly": false,
"Containers": {
"1d321acce71c3cde5f3c31f48f5844924b6a15c8a5ff4364f7c45a2369140363": {
"Name": "portainer_portainer.1.6sni6kg0mh7f2g1ngefj0pwke",
"EndpointID": "4a21396de7ecd673cc69f3dacf92e20b3544e8a247ec1175843ebe3221020a73",
"MacAddress": "02:42:0a:00:01:05",
"IPv4Address": "10.0.1.5/24",
"IPv6Address": ""
},
"36116f03bb4cae6328e66a1bb58d57fd73c24f2b7b4ee3e5223040a50896fba4": {
"Name": "traefik_traefik.1.qfnkxhjptjhjc3u77zp9erfy7",
"EndpointID": "58517905ec50479b23a3d1435f9ae97941524ecd73f2bbc3844aafb906478863",
"MacAddress": "02:42:0a:00:01:1d",
"IPv4Address": "10.0.1.29/24",
"IPv6Address": ""
},
"df674f10547d23eca3bb6f495965f08e8f1cabbe663eff3b67f7cdb90e29d99b": {
"Name": "whoami_whoami.3.l4bl4483mk2zlhxrtqewmldl8",
"EndpointID": "77239ba8e1b9050c18dd00773df6bda90891ca1184f880b308cb649e87036d66",
"MacAddress": "02:42:0a:00:01:0b",
"IPv4Address": "10.0.1.11/24",
"IPv6Address": ""
},
"lb-traefik-public": {
"Name": "traefik-public-endpoint",
"EndpointID": "17152c3ebc528ce5c4b0aacd240e1b92fc5dcc416ce48696795efc195e418592",
"MacAddress": "02:42:0a:00:01:03",
"IPv4Address": "10.0.1.3/24",
"IPv6Address": ""
}
},
"Options": {
"com.docker.network.driver.overlay.vxlanid_list": "4097"
},
"Labels": {},
"Peers": [
{
"Name": "169a1a886742",
"IP": "207.154.xxx.xxx"
},
{
"Name": "6fb47e278ac1",
"IP": "109.241.xxx.xxx"
}
]
}
]
gbaranski@galio > docker network inspect trafik-public
[
{
"Name": "traefik-public",
"Id": "xkpqua30neu7abr9ldw86klfr",
"Created": "2022-09-04T23:19:05.345265356Z",
"Scope": "swarm",
"Driver": "overlay",
"EnableIPv6": false,
"IPAM": {
"Driver": "default",
"Options": null,
"Config": [
{
"Subnet": "10.0.1.0/24",
"Gateway": "10.0.1.1"
}
]
},
"Internal": false,
"Attachable": false,
"Ingress": false,
"ConfigFrom": {
"Network": ""
},
"ConfigOnly": false,
"Containers": {
"1c3125187edfe1ee371190cccdb25dd21dbfc9f1e8372038f4ebd23beda2c217": {
"Name": "whoami_whoami.2.vc5w3es8d6a72byolgcc0mdxy",
"EndpointID": "77046f345f6747e08ad86ddfcf8028093ca9b3d3fda38464e274518e72681af7",
"MacAddress": "02:42:0a:00:01:1b",
"IPv4Address": "10.0.1.27/24",
"IPv6Address": ""
},
"lb-traefik-public": {
"Name": "traefik-public-endpoint",
"EndpointID": "834eda51dac95ccdfcd0189591e54eb72905405c0b649499b701f532888c52f9",
"MacAddress": "02:42:0a:00:01:1c",
"IPv4Address": "10.0.1.28/24",
"IPv6Address": ""
}
},
"Options": {
"com.docker.network.driver.overlay.vxlanid_list": "4097"
},
"Labels": {},
"Peers": [
{
"Name": "6fb47e278ac1",
"IP": "109.241.xxx.xxx"
},
{
"Name": "169a1a886742",
"IP": "207.154.xxx.xxx"
},
{
"Name": "fbc321e19b68",
"IP": "10.0.0.28"
}
]
}
]
如您所見，來自Peers的docker network inspect中的對等galio還包括一些 IP 為 10.0.0.28 的對等點，這是其在子網中的地址。
gbaranski@galio > ip a
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000
link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
inet 127.0.0.1/8 scope host lo
valid_lft forever preferred_lft forever
inet6 ::1/128 scope host
valid_lft forever preferred_lft forever
2: enp0s3: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 9000 qdisc mq state UP group default qlen 1000
link/ether 02:00:17:06:b5:bd brd ff:ff:ff:ff:ff:ff
inet 10.0.0.28/24 metric 100 brd 10.0.0.255 scope global enp0s3
valid_lft forever preferred_lft forever
inet6 fe80::17ff:fe06:b5bd/64 scope link
valid_lft forever preferred_lft forever
3: docker_gwbridge: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP group default
link/ether 02:42:6a:ea:b0:9d brd ff:ff:ff:ff:ff:ff
inet 172.18.0.1/16 brd 172.18.255.255 scope global docker_gwbridge
valid_lft forever preferred_lft forever
inet6 fe80::42:6aff:feea:b09d/64 scope link
valid_lft forever preferred_lft forever
4: docker0: <NO-CARRIER,BROADCAST,MULTICAST,UP> mtu 1500 qdisc noqueue state DOWN group default
link/ether 02:42:21:37:86:c0 brd ff:ff:ff:ff:ff:ff
inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0
valid_lft forever preferred_lft forever
10: veth4d81564@if9: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue master docker_gwbridge state UP group default
link/ether 4e:8d:3a:5c:50:76 brd ff:ff:ff:ff:ff:ff link-netnsid 1
inet6 fe80::4c8d:3aff:fe5c:5076/64 scope link
valid_lft forever preferred_lft forever
20: vethfa32c51@if19: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue master docker_gwbridge state UP group default
link/ether ae:a2:15:c7:30:40 brd ff:ff:ff:ff:ff:ff link-netnsid 4
inet6 fe80::aca2:15ff:fec7:3040/64 scope link
valid_lft forever preferred_lft forever
27: vethd70f359@if26: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue master docker_gwbridge state UP group default
link/ether e6:a9:2f:1c:9a:fd brd ff:ff:ff:ff:ff:ff link-netnsid 7
inet6 fe80::e4a9:2fff:fe1c:9afd/64 scope link
valid_lft forever preferred_lft forever
1 個解決方案
解決方案1
0 2022-09-04 23:55:28
解決了
我必須使用--advertise-addr標志加入群。 所以例如
docker swarm join --token <token> 207.154.xxx.xxx:2377 --advertise-addr 130.61.xxx.xxx
在docker中使用覆蓋使用多主機網絡功能需要swarm
[英]Is swarm required for using multi-host networking feature using overlay in docker
當我在私有 AWS 實例中克隆我的 bitbucket 存儲庫時，網絡無法訪問
[英]Network is unreachable when I clone my bitbucket repo in private AWS instance
聲明:本站的技術帖子網頁，遵循CC BY-SA 4.0協議，如果您需要轉載，請注明本站網址或者原文地址。任何問題請咨詢:yoyou2525@163.com.