stackoom
  简体   繁体   中英

Connecting to host's secondary IP from inside docker container

 原文  2018-07-12 18:30:26  9   1    amazon-web-services/ ubuntu/ docker/ networking

Question

I'm migrating an existing AWS (ubuntu) + Docker configuration to start using an Elastic Network Interface, so that I have persistent IPs in DNS. At boot time of the instance, I attach the ENI and it works great: 

  aws ec2 attach-network-interface --network-interface-id ${ENI} --instance-id ${INSTANCE_ID} --region ${REGION} --device-index 1

  ETH1=/etc/network/interfaces.d/eth1.cfg
  echo "auto eth1" > ${ETH1}
  echo "iface eth1 inet dhcp" >> ${ETH1}
  echo " up ip route add default via ${INSTANCE_SUBNET}.1 dev eth1 tab 2" >> ${ETH1}
  echo " up ip rule add from ${IP2}/32 tab 2" >> ${ETH1}
  echo " up ip rule add to ${IP2}/32 tab 2" >> ${ETH1}
  echo " up ip route flush cache" >> ${ETH1}
  ifup eth1

I'm successfully able to ping and ssh to the ENI address on eth1 remotely as well as locally on the host. It all works great. Except... Docker. From inside a container, I can connect to the host's eth0 interface, but not eth1. (For the purpose of this post, eth0==.207 and eth1==.213 .)

On the host, which boots up as .207: 

[root@10.180.11.207 ~]$ ping 10.180.11.207
PING 10.180.11.207 (10.180.11.207) 56(84) bytes of data.
64 bytes from 10.180.11.207: icmp_seq=1 ttl=64 time=0.028 ms
64 bytes from 10.180.11.207: icmp_seq=2 ttl=64 time=0.039 ms
^C
--- 10.180.11.207 ping statistics ---
2 packets transmitted, 2 received, 0% packet loss, time 999ms
rtt min/avg/max/mdev = 0.028/0.033/0.039/0.008 ms
[root@10.180.11.207 ~]$ ping 10.180.11.213
PING 10.180.11.213 (10.180.11.213) 56(84) bytes of data.
64 bytes from 10.180.11.213: icmp_seq=1 ttl=64 time=0.030 ms
64 bytes from 10.180.11.213: icmp_seq=2 ttl=64 time=0.037 ms
64 bytes from 10.180.11.213: icmp_seq=3 ttl=64 time=0.037 ms
^C
--- 10.180.11.213 ping statistics ---
3 packets transmitted, 3 received, 0% packet loss, time 1998ms
rtt min/avg/max/mdev = 0.030/0.034/0.037/0.007 ms

Inside docker: 

[root@10.180.11.207 ~]$ docker run -t -i ubuntu /bin/bash
root@42e0b7d1d484:/# (apt update && apt install -y iputils-ping) > /dev/null
root@42e0b7d1d484:/# ping 10.180.11.207
PING 10.180.11.207 (10.180.11.207) 56(84) bytes of data.
64 bytes from 10.180.11.207: icmp_seq=1 ttl=64 time=0.047 ms
64 bytes from 10.180.11.207: icmp_seq=2 ttl=64 time=0.053 ms
^C
--- 10.180.11.207 ping statistics ---
2 packets transmitted, 2 received, 0% packet loss, time 999ms
rtt min/avg/max/mdev = 0.047/0.050/0.053/0.003 ms
root@42e0b7d1d484:/# ping 10.180.11.213
PING 10.180.11.213 (10.180.11.213) 56(84) bytes of data.
^C
--- 10.180.11.213 ping statistics ---
13 packets transmitted, 0 received, 100% packet loss, time 12095ms

root@42e0b7d1d484:/#

Back on the host, it all looks ok to me... 

[root@10.180.11.207 ~]$ ifconfig -a
docker0   Link encap:Ethernet  HWaddr 02:42:70:51:aa:6a
          inet addr:172.17.0.1  Bcast:0.0.0.0  Mask:255.255.0.0
          inet6 addr: fe80::42:70ff:fe51:aa6a/64 Scope:Link
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
          RX packets:35525 errors:0 dropped:0 overruns:0 frame:0
          TX packets:42531 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:0
          RX bytes:2099259 (2.0 MB)  TX bytes:120243109 (120.2 MB)

eth0      Link encap:Ethernet  HWaddr 06:8c:bb:46:24:10
          inet addr:10.180.11.207  Bcast:10.180.11.255  Mask:255.255.255.0
          inet6 addr: fe80::48c:bbff:fe46:2410/64 Scope:Link
          UP BROADCAST RUNNING MULTICAST  MTU:9001  Metric:1
          RX packets:132933 errors:0 dropped:0 overruns:0 frame:0
          TX packets:49069 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1000
          RX bytes:182046681 (182.0 MB)  TX bytes:4131332 (4.1 MB)

eth1      Link encap:Ethernet  HWaddr 06:b9:f0:13:06:4a
          inet addr:10.180.11.213  Bcast:10.180.11.255  Mask:255.255.255.0
          inet6 addr: fe80::4b9:f0ff:fe13:64a/64 Scope:Link
          UP BROADCAST RUNNING MULTICAST  MTU:9001  Metric:1
          RX packets:33623 errors:0 dropped:0 overruns:0 frame:0
          TX packets:32080 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1000
          RX bytes:1877291 (1.8 MB)  TX bytes:14361232 (14.3 MB)

lo        Link encap:Local Loopback
          inet addr:127.0.0.1  Mask:255.0.0.0
          inet6 addr: ::1/128 Scope:Host
          UP LOOPBACK RUNNING  MTU:65536  Metric:1
          RX packets:244 errors:0 dropped:0 overruns:0 frame:0
          TX packets:244 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1
          RX bytes:21188 (21.1 KB)  TX bytes:21188 (21.1 KB)

vethfd8cd11 Link encap:Ethernet  HWaddr aa:81:86:18:db:f4
          inet6 addr: fe80::a881:86ff:fe18:dbf4/64 Scope:Link
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
          RX packets:17230 errors:0 dropped:0 overruns:0 frame:0
          TX packets:18203 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:0
          RX bytes:1340186 (1.3 MB)  TX bytes:40751823 (40.7 MB)

nothing stands out to me in the route tables 

[root@10.180.11.207 ~]$ ip route list
default via 10.180.11.1 dev eth0
10.180.11.0/24 dev eth0  proto kernel  scope link  src 10.180.11.207
10.180.11.0/24 dev eth1  proto kernel  scope link  src 10.180.11.213
172.17.0.0/16 dev docker0  proto kernel  scope link  src 172.17.0.1

or iptables 

[root@10.180.11.207 ~]$ iptables --list

Chain INPUT (policy ACCEPT)
target     prot opt source               destination
ACCEPT     all  --  anywhere             anywhere
ACCEPT     all  --  anywhere             anywhere
ACCEPT     all  --  anywhere             anywhere

Chain FORWARD (policy ACCEPT)
target     prot opt source               destination
DOCKER-ISOLATION  all  --  anywhere             anywhere
DOCKER     all  --  anywhere             anywhere
ACCEPT     all  --  anywhere             anywhere             ctstate RELATED,ESTABLISHED
ACCEPT     all  --  anywhere             anywhere
ACCEPT     all  --  anywhere             anywhere

Chain OUTPUT (policy ACCEPT)
target     prot opt source               destination

Chain DOCKER (1 references)
target     prot opt source               destination

Chain DOCKER-ISOLATION (1 references)
target     prot opt source               destination
RETURN     all  --  anywhere             anywhere

I can see the packets on docker0. I don't see anything on eth0 (or eth1) but I'm assuming it should be loopback anyway? 

[root@10.180.11.207 ~]$ tcpdump -n -i docker0 port not 22
tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
listening on docker0, link-type EN10MB (Ethernet), capture size 262144 bytes
18:12:44.497216 ARP, Request who-has 172.17.0.2 tell 172.17.0.1, length 28
18:12:44.497274 ARP, Reply 172.17.0.2 is-at 02:42:ac:11:00:02, length 28
18:12:46.522555 IP 172.17.0.2 > 10.180.11.207: ICMP echo request, id 297, seq 1, length 64
18:12:46.522590 IP 10.180.11.207 > 172.17.0.2: ICMP echo reply, id 297, seq 1, length 64
18:12:47.521557 IP 172.17.0.2 > 10.180.11.207: ICMP echo request, id 297, seq 2, length 64
18:12:47.521595 IP 10.180.11.207 > 172.17.0.2: ICMP echo reply, id 297, seq 2, length 64
18:12:48.521272 IP 172.17.0.2 > 10.180.11.207: ICMP echo request, id 297, seq 3, length 64
18:12:48.521311 IP 10.180.11.207 > 172.17.0.2: ICMP echo reply, id 297, seq 3, length 64
18:12:57.207506 IP 172.17.0.2 > 10.180.11.213: ICMP echo request, id 298, seq 1, length 64
18:12:58.215426 IP 172.17.0.2 > 10.180.11.213: ICMP echo request, id 298, seq 2, length 64
18:12:59.223387 IP 172.17.0.2 > 10.180.11.213: ICMP echo request, id 298, seq 3, length 64
18:13:00.231388 IP 172.17.0.2 > 10.180.11.213: ICMP echo request, id 298, seq 4, length 64
18:13:01.239439 IP 172.17.0.2 > 10.180.11.213: ICMP echo request, id 298, seq 5, length 64
18:13:02.247412 IP 172.17.0.2 > 10.180.11.213: ICMP echo request, id 298, seq 6, length 64
18:13:03.255409 IP 172.17.0.2 > 10.180.11.213: ICMP echo request, id 298, seq 7, length 64

routing decisions look ok: 

[root@10.180.11.207 ~]$ ip route get 10.180.11.213 from 172.17.0.1
local 10.180.11.213 from 172.17.0.1 dev lo
    cache <local>
[root@10.180.11.207 ~]$ ip route get 10.180.11.207 from 172.17.0.1
local 10.180.11.207 from 172.17.0.1 dev lo
    cache <local>
[root@10.180.11.207 ~]$ ip route get 10.180.11.213 from 10.180.11.207
local 10.180.11.213 from 10.180.11.207 dev lo
    cache <local>

Any ideas? I've exhausted every trick I know.

1 answers

solution1
 1  2018-07-12 19:01:19

I think the issue lies in the way the network bridge is configured. docker0 is a bridge to eth0. You need another bridge to eth1 and then attach that bridge to the container as another network.

An example may be: 

docker network create -d bridge \
--subnet=172.30.0.32/24 \
--opt com.docker.network.bridge.enable_ip_masquerade=true \
--opt com.docker.network.bridge.host_binding_ipv4=10.180.11.213 \
--opt com.docker.network.bridge.name=docker1 \
docker1

The important part is the com.docker.network.bridge.host_binding_ipv4 which is the eth1 IP.

Then connect the container to that bridge: 

docker network connect docker1 container-name

暂无
暂无

The technical post webpages of this site follow the CC BY-SA 4.0 protocol. If you need to reprint, please indicate the site URL or the original address.Any question please contact:yoyou2525@163.com.

Related Question Error connecting to RDS Postgreql DB from inside Docker container Assign Host IP-Address to Docker Container how to get aws credentials from host workstation, inside docker container Cannot connect to RDS from inside a docker container, I can from the host, and I can from a local docker container Connecting to RDS Mysql from docker container with Wordpress Docker AWS access container from Ip Assuming a Sagemaker Notebook host's role from Docker container with network mode = bridge SSH into a docker container from another container on a different host Docker on AWS - Environment Variables not inheriting from host to container How to mount file from host to docker container on ECS
Related Tags
 
粤ICP备18138465号  © 2020-2024 STACKOOM.COM