kubernetes 无法通过 ip 从 pod 内部访问其他机器

Question

kubernetes 无法通过 ip 从 pod 内部访问其他机器

kubectl exec dnsutils  -it /bin/bash
root@dnsutils:/# ping 10.116.197.60
PING 10.116.197.60 (10.116.197.60) 56(84) bytes of data.

但它适用于机器

ping 10.116.197.60
PING 10.116.197.60 (10.116.197.60) 56(84) bytes of data.
64 bytes from 10.116.197.60: icmp_seq=1 ttl=64 time=0.854 ms
64 bytes from 10.116.197.60: icmp_seq=2 ttl=64 time=0.906 ms
...

并适用于 docker 容器

docker exec -it bind /bin/bash
root@0f356bf598c5:/# ping 10.116.197.60
PING 10.116.197.60 (10.116.197.60): 56 data bytes
64 bytes from 10.116.197.60: icmp_seq=0 ttl=63 time=1.172 ms
64 bytes from 10.116.197.60: icmp_seq=1 ttl=63 time=1.007 ms
64 bytes from 10.116.197.60: icmp_seq=2 ttl=63 time=1.260 ms
64 bytes from 10.116.197.60: icmp_seq=3 ttl=63 time=1.307 ms
64 bytes from 10.116.197.60: icmp_seq=4 ttl=63 time=1.118 ms
64 bytes from 10.116.197.60: icmp_seq=5 ttl=63 time=1.023 ms
...

在 pod 中使用 tracerouter

/ # traceroute -n -m 5 -q 4 -w 3 10.116.197.60
traceroute to 10.116.197.60 (10.116.197.60), 5 hops max, 46 byte packets
 1  10.233.0.1  0.008 ms  0.005 ms  0.004 ms  0.004 ms
 2  *  *  *  *
 3  *  *  *  *
 4  *  *  *  *
 5  *  *  *  *

/ # ip addr
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN qlen 1000
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
    inet 127.0.0.1/8 scope host lo
       valid_lft forever preferred_lft forever
3: eth0@if64: <BROADCAST,MULTICAST,UP,LOWER_UP,M-DOWN> mtu 1450 qdisc noqueue state UP 
    link/ether 82:71:94:c7:fe:90 brd ff:ff:ff:ff:ff:ff
    inet 10.233.0.139/24 brd 10.233.0.255 scope global eth0
       valid_lft forever preferred_lft forever

看起来像 10.233.0.1 中的一些错误，但我不知道为什么

kubernetes 版本：1.20 网络：法兰绒模式：ipvs

经过一些测试，在没有 snat 的情况下 ping 其他机器。

09:20:24.997764 IP 10.233.0.156 > 10.116.197.60: ICMP echo request, id 149, seq 187, length 64
09:20:24.997888 IP 10.116.197.60 > 10.233.0.156: ICMP echo reply, id 149, seq 187, length 64
09:20:26.021795 IP 10.233.0.156 > 10.116.197.60: ICMP echo request, id 149, seq 188, length 64
09:20:26.021876 IP 10.116.197.60 > 10.233.0.156: ICMP echo reply, id 149, seq 188, length 64
09:20:27.045738 IP 10.233.0.156 > 10.116.197.60: ICMP echo request, id 149, seq 189, length 64
09:20:27.045825 IP 10.116.197.60 > 10.233.0.156: ICMP echo reply, id 149, seq 189, length 64

吊舱的ip 10.233.0.156是ip，机器无法回复

Answer 1

添加 iptables 的规则来抓取 pod ip

iptables -t nat -A POSTROUTING -s 10.233.0.0/24 -j MASQUERADE

10.233.0.0/24是--pod-network-cidr的值

yum install iptables-services -y
iptables -F
iptables -P INPUT ACCEPT
iptables -P FORWARD ACCEPT
iptables -t nat -A POSTROUTING -s 10.233.0.0/24 -j MASQUERADE
service iptables save
systemctl enable iptables.service