Access host docker-machine from within container

Question

I have an image that I'm using to run my CI/CD builds (using GitLab CE). I'd like to deploy my app doing something like this from within the container:

eval "$(docker-machine env manager)"
sudo docker stack deploy --compose-file docker-stack.yml web

However, I'd like the docker-machine to access machines defined on the host system since the container will be destroyed and I don't want to include access details in the image.

I've tried a few things

Accessing the Remote Host via docker-machine

• Create the docker-machine on the host and mount the MACHINE_STORAGE_PATH so that it is available to the container
• Connect to the remote docker-machine manually from within the container and setting the MACHINE_STORAGE_PATH equal to a mounted volume
• Mounting the docker socket

In both cases, I can see the machine storage is persisted, but whenever I create a new container and run docker-machine ls none of the machines are listed.

Accessing the Remote Host via DOCKER_HOST

• Forward the remote machine docker port to the host docker port docker-machine ssh manager-1 -N -L 2376:localhost:2376
• export DOCKER_HOST=:2376
• Tell docker to use the same certs that are used by docker-machine: export DOCKER_TLS_VERIFY=1 and export DOCKER_CERT_PATH=/Users/me/.docker/machine/machines/manager-‌​1
• Test with docker info

This gives me error during connect: Get https://localhost:2376/v1.26/info: x509: certificate signed by unknown authority

Any ideas on how I can perform a remote deployment from within a container?

Thanks

EDIT

Here is a diagram to try and help better communicate the scenario.

Answer 1

Don't use docker-machine for this.

Docker-machine stores files in $HOME/.docker/machine, so when you restart with a fresh copy of this folder, all previously defined machines will be removed. You could store this folder as a volume, but there's a much easier way for your purposes.

The solution is to mount the docker socket, and either as root or from a user with the same gid as the docker socket (note that group names themselves inside and outside the container may not match, so gid is important), run your docker ... commands as normal. You can skip the docker-machine eval completely since you are running the commands against the local docker socket.

If you need to run commands remotely, I find it easier to define the DOCKER_HOST and DOCKER_TLS_VERIFY variables manually rather than using docker-machine .

Answer 2

In case you want to communicate from your CI container to the Docker host you can simply mount the Docker socket when starting the CI container:

docker run -v /var/run/docker.sock:/var/run/docker.sock <gitlab-image>

Now you can run docker commands on the host from within the CI container.