如何将文件从一个 docker 容器复制到另一台主机上的另一个容器？

Question

What is the best way to copy files from one docker container to another on a different host? we ran into couple of issues while trying to do this.

Issue #1: we provisioned an overlay network and attached containers to it. The containers can ping each other using IP address but not using the hostname provided using the -h flag when creating the container

Container 1:

root@rca-ord:/# cat /etc/hosts
127.0.0.1   localhost
::1 localhost ip6-localhost ip6-loopback
fe00::0 ip6-localnet
ff00::0 ip6-mcastprefix
ff02::1 ip6-allnodes
ff02::2 ip6-allrouters
10.0.29.10  rca-ord

Container 2:

root@ica-ord:/# cat /etc/hosts
127.0.0.1   localhost
::1 localhost ip6-localhost ip6-loopback
fe00::0 ip6-localnet
ff00::0 ip6-mcastprefix
ff02::1 ip6-allnodes
ff02::2 ip6-allrouters
10.0.29.11  ica-ord

ping using hostname fails

root@ica-ord:/# ping rca-ord
ping: unknown host rca-ord

ping using IP address works

root@ica-ord:/# ping 10.0.29.10
PING 10.0.29.10 (10.0.29.10) 56(84) bytes of data.
64 bytes from 10.0.29.10: icmp_seq=1 ttl=64 time=0.684 ms
64 bytes from 10.0.29.10: icmp_seq=2 ttl=64 time=0.434 ms
64 bytes from 10.0.29.10: icmp_seq=3 ttl=64 time=0.731 ms
64 bytes from 10.0.29.10: icmp_seq=4 ttl=64 time=0.672 ms
64 bytes from 10.0.29.10: icmp_seq=5 ttl=64 time=0.545 ms
64 bytes from 10.0.29.10: icmp_seq=6 ttl=64 time=1.25 ms
^C
--- 10.0.29.10 ping statistics ---
6 packets transmitted, 6 received, 0% packet loss, time 5010ms
rtt min/avg/max/mdev = 0.434/0.719/1.251/0.258 ms

Looks like we need to run some DNS server. How? what are the details?

Issue #2: we tried to scp using the IP address and it asks for a password - likely the root password as that is the username under which we are logged in. we don't know root password.

root@ica-ord:/# scp 10.0.29.10:test-file.txt .
The authenticity of host '10.0.29.10 (10.0.29.10)' can't be established.
ECDSA key fingerprint is SHA256:Bdnjhvxlk1ILzWpa8tvu+YLaHRqh9XBftIoeLSpKLBQ.
Are you sure you want to continue connecting (yes/no)? yes
Warning: Permanently added '10.0.29.10' (ECDSA) to the list of known hosts.
root@10.0.29.10's password: 

we are thinking there ought to be a simple way to copy files between containers on multiple hosts. is there any?

Answer 1

Docker doesn't provide anything magical to make this easy. Typically Docker-packaged components don't communicate with each other by sharing files; instead they talk to each other using network protocols like HTTP, or by using a message-bus system like RabbitMQ.

This problem isn't fundamentally different from taking two non-Docker server processes that used to run on the same host and splitting them on to different hosts, except that Docker's filesystem isolation means that just getting the file on to the other host doesn't necessarily make it visible to the other container.

A typical architecture I've seen around this looks like this:

1. You have some sort of shared networked file-oriented storage layer (an NFS server, Amazon S3, ...). Both services have the required setup to communicate with it.

2. The producing service writes the file to the storage layer.

3. The producing service contacts the consuming service (by an HTTP connection; to a shared RabbitMQ queue; either knowing the other service's server's hostname or using a service-discovery system like Consul or Kubernetes Service objects to find it) and tells it where the file is.

4. The consuming service reads the file and processes it.

This setup has the advantages that you don't need to deal with N services trying to set up authenticated scp to N other services (or even M other hosts), you don't have to deal with the complexities of Docker volume sharing (and concurrent access and permission management), and it still works even if a service moves to a different host and doesn't immediately have a local copy of the file.