I have read the some documents before asking this question:
And I am confused about the following two parts:
Their behavior is completely opposite:
For bind mounts:
If you bind-mount into a non-empty directory on the container, the directory's existing contents are obscured by the bind mount.
For named volume and anonymous volumes:
If you start a container which creates a new volume, as above, and the container has files or directories in the directory to be mounted (such as /app/ above), the directory's contents are copied into the volume. The container then mounts and uses the volume, and other containers which use the volume also have access to the pre-populated content.
My question is, if I use a VOLUME
command in dockerfile to create a named volume anonymous volumes, and at the same time, use a bind mount to mount a non exist path into the container, what goes on behind the scenes?
For example:
# in docker file
VOLUME /path/in/container
# when run container
docker run -v /not-exist-dir/in/host:/path/in/container ... /< image >
I have do some test and the result is:
/var/lib/docker/volumes/
So,
what happened behind this scenario?
Let me give a more specific example:
Here is the dockerfile of the openfrontier/gerrit image , we can see at the end of the docker file, there is a VOLUME command:
VOLUME $GERRIT_SITE
which will actually create an anonymous volumes on the host and mount it to /var/gerrit/review_site
(value of GERRIT_SITE) in the container when any container is created from this image like this:
docker run -dit --name gerrit openfrontier/gerrit:2.15.3
after that, I can see the anonymous volumes under /var/lib/docker/volumes/
and I can use docker volume ls
to see its name be4538dbf3a51da463391c6eca9714fb6dd0c11379f1e2918f74c33d56633f00
, and also with docker inspect gerrit
command, we can see:
"Mounts": [
{
"Type": "volume",
"Name": "be4538dbf3a51da463391c6eca9714fb6dd0c11379f1e2918f74c33d56633f00",
"Source": "/var/lib/docker/volumes/be4538dbf3a51da463391c6eca9714fb6dd0c11379f1e2918f74c33d56633f00/_data",
"Destination": "/var/gerrit/review_site",
"Driver": "local",
"Mode": "",
"RW": true,
"Propagation": ""
}
],
and there are some file under these two folder, which are created by the container after it running.
Till now, this is a normal example for create an anonymous volumes by using VOLUME command.
However, if I run this container with the following command:
docker run -dit --name gerrit -v /home/test:/var/gerrit/review_site openfrontier/gerrit:2.15.3
where /home/test
is not exist on the host, then no anonymous volumes is created, instead, /home/test
folder is created and it is not empty!!!, the mount info is as follows:
"Mounts": [
{
"Type": "bind",
"Source": "/home/test",
"Destination": "/var/gerrit/review_site",
"Mode": "",
"RW": true,
"Propagation": "rprivate"
}
],
I think bind mounts is take effects here but I just do not understand why the path on the host is not empty(since bind-mount into a non-empty directory on the container, the directory's existing contents are obscured by the bind mount.)
I also test mounting a not empty folder to the container by put some file under "/home/test" before I use docker run
to start a container, the result is the original file was kept and new files was added too.
I know the examples above are not good practice to use docker volume, But I just wondering what happened behind.
The volume declaration in the Dockerfile sets a bit of metadata on the image that tells docker run
to define an anonymous volume at that location any time a container is made from that image. This is not a named volume or host mount, but it has a lot in common with both (they are all bind mounts by default, and anonymous volumes are listed in docker volume ls
along with named volumes).
However, when you specify a host mount to the same container directory, that takes precedence (two volume mounts to the same directory inside the container will not happen). Any modifications to that directory by the commands running inside the container will be visible on the host, but the directory itself will not be initialized by the image contents.
If you want to initialize a host directory with image contents, you can use a named volume that performs a bind mount. There are a few differences in behaviour from the host mount. Mainly the directory must exist in advance, and inside a compose file you must use absolute paths instead of relative ones. I've got the syntax for that in my presentation here:
https://sudo-bmitch.github.io/presentations/dc2018eu/tips-and-tricks-of-the-captains.html#48
There are some mixed up things in your question, and since it's not always intuitive, I'll try to make it clearer through example.
Notes :
Dockerfile
FROM ubuntu
VOLUME /tmp/dockerfilevolumefromnowhere
RUN mkdir -p /dir/created/from/container
RUN touch /dir/created/from/container/emptyFile.txt
VOLUME /dir/created/from/container
CMD "sh"
let's remove all previous tests data
BE AWARE THAT IT WILL REMOVE ALL VOLUMES THAT ARE NOT USED ANYMORE, including those that you may have created previously.
If you have any running containers (especially if they use volumes), please stop them for the purpose of these tests.
root@host:~# docker container prune
root@host:~# docker volume prune
root@host:~# rm -r /does/not/exit/within/host
root@host:~# rm -r /does/not/exit/within/host2
root@host:~# rm -r /tmp/dockerfilevolumefromnowhere
some image and volume creations and filling
root@host:~# docker build -t tmpcontainer .
root@host:~# docker volume create existingVolume
root@host:~# touch /var/lib/docker/volumes/existingVolume/_data/someExistingFile.txt
root@host:~# docker volume create existingVolume2
root@host:~# touch /var/lib/docker/volumes/existingVolume2/_data/anotherExistingFile.txt
root@host:~# docker volume create existingEmptyVolume
at this time you have 3"existing volumes":
root@host:~# docker volume ls
DRIVER VOLUME NAME
local existingVolume
local existingVolume2
local existingEmptyVolume
Let's now run our container
docker run -it \
-v /does/not/exit/within/host:/does/not/exist/within/container \
-v /does/not/exit/within/host2:/sbin \
-v /tmp:/again/another/does/not/exist/within/container \
-v /tmp:/tmp/ \
-v newVolume:/another/does/not/exist/within/container \
-v newVolume2:/bin \
-v existingVolume:/new/path/on/container \
-v existingVolume2:/usr \
-v existingEmptyVolume:/var \
bash: groups: command not found # that's normal. you overrided the /usr... see beyond
Now, we are connected on our newly created container.
Let's see what we have on both container and host :
# -v /does/not/exit/within/host:/does/not/exist/within/container
#no one exists on both sides : both directories are created, and are now bound to each other
root@host:~# ll /does/not/exit/within/host
.
root@ffb82b56d64b:/# ll /does/not/exist/within/container
.
# -v /does/not/exit/within/host:/sbin
# the directory on host is created, and the one on container is erased with this new one. Both are now bound to each other
root@host:~# ll /does/not/exit/within/host2
.
root@ffb82b56d64b:/# ll /sbin
.
# -v /tmp:/again/another/does/not/exist/within/container
# the path on host exists, and the path on container will be created and will hold the content on container (they are bound to each other)
root@host:~# ll /tmp
-rw------- 1 root root 65536 Feb 19 11:11 one.txt
-rw------- 1 root root 65536 Feb 19 11:11 two.yml
root@ffb82b56d64b:/# ll /again/another/does/not/exist/within/container
-rw------- 1 root root 65536 Feb 19 11:11 one.txt
-rw------- 1 root root 65536 Feb 19 11:11 two.yml
# -v /tmp:/tmp
# the path on host exists, so all its content will replace the previously path on container
# there were some data on container's /tmp, but they are replaced with hosts ones
root@host:~# ll /tmp
-rw------- 1 root root 65536 Feb 19 11:11 one.txt
-rw------- 1 root root 65536 Feb 19 11:11 two.yml
root@ffb82b56d64b:/# ll /tmp
-rw------- 1 root root 65536 Feb 19 11:11 one.txt
-rw------- 1 root root 65536 Feb 19 11:11 two.yml
# -v newVolume:/another/does/not/exist/within/container \
# the newVolume does not exist, so it will be created and bound to path on container.
# since the path on container is new, it will be created empty
root@host:~# ll /var/lib/docker/volumes/newVolume/_data/
..
root@ffb82b56d64b:/# ll /another/does/not/exist/within/container
..
# -v newVolume2:/bin \
# once again, a volume will be created, but since it matches an existing path on container, it will hold all the content of it (no erasal!)
root@host:~# ll /var/lib/docker/volumes/newVolume2/_data/
<all the content of Ubuntu's /bin from within container>
root@ffb82b56d64b:/# ll /bin
<whole expected content of /bin on Ubuntu>
# -v existingVolume:/new/path/on/container \
# the volume exists, and it -and all files within- will be bound to a newly created path on container
root@host:~# ll /var/lib/docker/volumes/existingVolume/_data/
-rw------- 1 root root 65536 Feb 19 11:11 someExistingFile.txt
root@ffb82b56d64b:/# ll /new/path/on/container
-rw------- 1 root root 65536 Feb 19 11:11 someExistingFile.txt
# -v existingVolume2:/usr \
# the volume exists, so does the path on container. It will replace the existing path (and thus erase the former files there) and will be bound to this replaced path.
root@host:~# ll /var/lib/docker/volumes/existingVolume2/_data/
-rw------- 1 root root 65536 Feb 19 11:11 anotherExistingFile.txt
root@ffb82b56d64b:/# ll /usr
-rw------- 1 root root 65536 Feb 19 11:11 anotherExistingFile.txt
# -v existingEmptyVolume:/var \
# the volume exists, but is empty. the path on container exists in the container and is not empty. In this case, the path on container will not be erased and this will act as a new volume.
root@host:~# ll /var/lib/docker/volumes/existingEmptyVolume/_data/
drwxr-xr-x 2 root root 4096 Apr 24 2018 backups/
drwxr-xr-x 5 root root 4096 Feb 19 14:36 cache/
drwxr-xr-x 7 root root 4096 Feb 19 14:36 lib/
drwxrwsr-x 2 root staff 4096 Apr 24 2018 local/
lrwxrwxrwx 1 root root 9 Nov 12 21:54 lock -> /run/lock/
drwxr-xr-x 3 root root 4096 Feb 19 14:36 log/
drwxrwsr-x 2 root mail 4096 Nov 12 21:54 mail/
drwxr-xr-x 2 root root 4096 Nov 12 21:54 opt/
lrwxrwxrwx 1 root root 4 Nov 12 21:54 run -> /run/
drwxr-xr-x 2 root root 4096 Feb 19 14:36 spool/
drwxrwxrwt 2 root root 4096 Nov 12 21:56 tmp/
root@ffb82b56d64b:/# ll /var
drwxr-xr-x 2 root root 4096 Apr 24 2018 backups/
drwxr-xr-x 5 root root 4096 Feb 19 14:36 cache/
drwxr-xr-x 7 root root 4096 Feb 19 14:36 lib/
drwxrwsr-x 2 root staff 4096 Apr 24 2018 local/
lrwxrwxrwx 1 root root 9 Nov 12 21:54 lock -> /run/lock/
drwxr-xr-x 3 root root 4096 Feb 19 14:36 log/
drwxrwsr-x 2 root mail 4096 Nov 12 21:54 mail/
drwxr-xr-x 2 root root 4096 Nov 12 21:54 opt/
lrwxrwxrwx 1 root root 4 Nov 12 21:54 run -> /run/
drwxr-xr-x 2 root root 4096 Feb 19 14:36 spool/
drwxrwxrwt 2 root root 4096 Nov 12 21:56 tmp/
Note that now, the previously existing and newly created volume can be used for other containers (the same way you just used previously existing ones here above).
Eventually, all that we have left to cover all our uses cases are the volumes created from the Dockerfile . Let's look at the content of /var/lib/docker/volumes. Remember we have cleaned it before performing our tests, so all those here match our tests.
root@host:~# ll /var/lib/docker/volumes
drwx------ 6 root root 4096 Feb 19 11:11 ./
drwx--x--x 14 root root 4096 Feb 14 09:50 ../
-rw------- 1 root root 65536 Feb 19 14:36 metadata.db #indeed
drwxr-xr-x 3 root root 4096 Feb 19 14:36 635af95ec06f8a44c22915005189bfb12d5bcf2e5ac97c25112d3e65a72546f4/ # anonymous container 1
drwxr-xr-x 3 root root 4096 Feb 19 14:36 897b28ec17275a3c3be184bb20b6314b38c1404e638080c8fe4fc36cae2f9f65/ # anonymous container 2
drwxr-xr-x 3 root root 4096 Feb 19 14:36 existingEmptyVolume/ #we created it (empty) before the run command
drwxr-xr-x 3 root root 4096 Feb 19 14:35 existingVolume/ #we created it before the run command
drwxr-xr-x 3 root root 4096 Feb 19 14:36 existingVolume2/ #we created it before the run command
drwxr-xr-x 3 root root 4096 Feb 19 14:36 newVolume/ #we created it during run command
drwxr-xr-x 3 root root 4096 Feb 19 14:36 newVolume2/ #we created it during run command
So what are those anonymous containers : they are actually the ones created from you dockerfile. Let's check their content.
root@host:~# ll /var/lib/docker/volumes/63eeedcb1aa2e4d8785cca409698371381558348ce19bc614d87da372901d224/_data/
-rw-r--r-- 1 root root 0 Feb 19 10:14 emptyFile.txt
root@host:~# ll /var/lib/docker/volumes/ea0ed5ff271cba03a8b7d35144b58e8da1b2e50b4e05c4cccda7f19b401d7f0b/_data/
..
See, they are holding whatever you put in them from the dockerfile. They are anonymous, so you cannot (or actually must not) use them from other containers because new ones will be created for every time you run a container from this image.
Note : You may have noticed some odd thing at this time:
root@host:~# ll /tmp/dockerfilevolumefromnowhere #YES : from HOST!
.. #it exists... but is not linked to the volume or the path in container in any way.
Here... don't ask : it is still puzzling me why it is created on host. I guess it's a bug, but this an issue that should be discussed on docker ML, not here. Anyway, you are not supposed to do it that way : mkdir before creating the volume over it!
So what we can see with all these exemples :
Note : that I am not talking about ownership and rights here. On purpose: that's another matter we may discuss later.
The technical post webpages of this site follow the CC BY-SA 4.0 protocol. If you need to reprint, please indicate the site URL or the original address.Any question please contact:yoyou2525@163.com.