Volumes in Docker inside Docker?

Question

I am running buildbot which is a CI tool on an EC2 machine. It's currently running as docker containers one for buildbot master and one for buildbot worker. Inside buildbot worker, I have to again run docker for building images and running containers.

After doing some research on how to best do this, I have mounted the docker sock file from the host machine to the buildbot worker container. Now from inside the buildbot worker, I am able to connect to the host docker daemon and use the build cache.

Main problem now is that inside the buildbot worker, I have a docker compose file in which for one service, I am mounting a file like this ./configs/my.cnf:/etc/my.cnf but it is failing. And doing some more research, it's because the configs/my.cnf is relative to the buildbot worker directory and since I am using the host docker daemon which resolves the files using the host paths, it is not able to find the file.

I am not able to figure out on how to best do this. There were some suggestions on using the data volumes for this, but I am not sure on how best to use those. Any idea on how we can do this?

Answer 1

Do you have any control over the creation of the buildbot worker? Can you control the buildbot worker directory.

export BUILD_BOT_DIR=$(mktemp -d) &&
    docker container create -v /var/run/docker.sock:/var/run/docker.sock -v ${BUILD_BOT_DIR}:${BUILD_BOT_DIR} -e BUILD_BOT_DIR ...

In this scenario, the path './configs/my:conf' points to the same file on both the container and the host.