How to use docker-compose environment variables to populate config file

Question

I'm new to docker-compose.
I'm trying to understand how can I pass environment variables via docker-compose to populate missing variables values in a config file during Dockerfile 's build stage, so in the run phase, I will run the service- rabbitmq with its configuration filled with the env variables values.

In docker-compose yml file I have an env_file: and also environment: , question how can I pass them forward to populate the config file.

Any help, an idea would be greatly appreciated with a simple example of course.

Example rabbitmq config file (with env variable which need to be filled with its value for $SSL_PORT):

[{rabbit,
  [
    {loopback_users, []},
    {ssl_listeners, [$SSL_PORT]},
    {ssl_options, [{cacertfile,"/etc/rabbitmq/ca/cacert.pem"},
                   {certfile,"/etc/rabbitmq/server/cert.pem"},
                   {keyfile,"/etc/rabbitmq/server/key.pem"},
                   {verify,verify_none},
                   {fail_if_no_peer_cert,false}]}
  ]}
].

Answer 1

The problem can be divided in 2 parts:

1. Inject the configuration values inside the container
2. Let the target software use such configuration values

The first problem is solved via the use of environment variables: as you stated you can either instruct docker to use env files ( --env-file file.env ) or specify single environment variables ( --env VARIABLE=value ) and docker will make them available inside the container.

The second problem has different solution based on the configuration framework used.

For instance if you are using something like Typesafe config or Spring you can just put a reference to the environment variable inside the configuration itself and the configuration framework will automatically expand it.

If instead you are configuring a third-party software which doesn't support environment variable expansion you will need to do some preprocessing before the target software gets executed.

This is usually done by creating a docker image with a custom entrypoint . This entrypoint is responsible of modifying the configuration of the target software expannding the known environment variables and then starting the software.

An example of such an entrypoint script could be:

#!/bin/sh

sed -i "s/\$SSL_PORT/$SSL_PORT/g" /etc/software.conf

exec $@

Modifying a configuration file in this way could be dangerous and error prone so take care when writing such scripts.

Update :

If all you want to do is to perform some pre-processing before the original entrypoint start you can follow this simple pattern: Let's say the original entrypoint is called docker-entrypoint.sh , all you need to do is to to create docker-entrypoint-pre.sh with this content:

#!/bin/bash

# Perfom all the needed preprocessing here...

# Invoke the original entrypoint passing the command and arguments
exec /docker-entrypoint.sh $@ 

If you build a docker image using your new entrypoint and start a container with:

docker run --rm test-image echo "This is a test"

What will happen is that docker-entrypoint-pre.sh will be called with arguments echo and This is a test . After the preprocessing is done the original entrypoint docker-entrypoint.sh will be called with the same arguments thus maintaining the behaviour of the original image.

Update2: In order to add your new entrypoint you have to create a new Docker image inheriting from the original. Lets say that the original image is rabbit:latest , you would need to create a Dockerfile with the following content:

FROM rabbit:latest

COPY docker-entrypoint-pre.sh /docker-entrypoint-pre.sh

ENTRYPOINT ["/docker-entrypoint-pre.sh"]

And then build your image with docker build -t myrabbit:latest . from the directory containing the Dockerfile and the docker-entrypoint-pre.sh .

At this point you have a new image myrabbit:latest with your preprocessing logic.

Answer 2

Assuming that your config file(s) are using the variable $SSL_PORT inside the container, and you want to pass environment variables from the host machine to be used on build, this can be done in your docker-compose.yml . YAML supports the use of environment variables, therefore, you can use environment: followed by an array of variables you wish to pass to the container during runtime.

For example:

version: '3'
services:
  your-app:
    build:
      context: .
      dockerfile: Dockerfile
    environment:
      - "SSL_PORT=${PORT_VARIABLE_FROM_HOST}"

To trouble shoot and make sure that your variables are in fact assigned inside your container run:

docker exec -it <container_name_or_id> printenv

This should list all of the environment variables inside your container