How to push multiple images needed for docker-compose to GitLab registry in GitLab CI?

Question

I recently got into CI/CD, and a good starting point for me was GitLab, since they provide an easy interface for that and i got started about what pipelines and stages are, but i have run into some kind of contradictory thought about GitLab CI running on Docker.

My app runs on Docker Compose. It contains (blah blah) that makes it easy to build & run containers. Each service in the Docker Compose creates a single Docker container, excepting the php-fpm one, which is able to do the thing called "horizontal scale", so I can scale it later.

I will use that Docker Compose for production, I am currently using it in development and I want to use it too in CI/CD pipelines.

However the.gitlab-ci.yml provides support for only one image, so I have to build it and push it to either their GitLab Registry or Docker Hub in order to pull it later in the CI/CD process.

How can I build my Docker Compose's service as a single image in order to push it to the Registry/Docker so I can pull it in the CI/CD?

My project contains a docker folder and a docker-compose.yml. In the docker folder, each service has its own separate directory (php-fpm, nginx, mysql, etc.) and each one (prepare yourself) contains a Dockerfile with build details, especially the php-fpm one (deps and libs are strong with this one)

Each service in the docker-compose.yml has a build context in each of their own folder.

If I was unclear, I can provide additonal info.

Answer 1

I think you search something like this

# .gitlab-ci.yml

image: docker

services:
  - docker:dind

build:
  script:
    - apk add --no-cache py-pip
    - pip install docker-compose
    - docker-compose up -d

Also good to know:

• In Docker, what's the difference between a container and an image?
• Building Docker images with GitLab CI/CD

Answer 2

You don't need dind to run a docker-compose stack. You can run multiple docker-compose up commands.

acceptance_testing:
  stage: test
  before_script:
    - docker-compose -p $CI_JOB_ID up -d
  script:
    - docker-compose -p $CI_JOB_ID exec -T /run/your/test/suite.sh
  after_script:
    - docker-compose -p $CI_JOB_ID down -v --remove-orphans || true

Answer 3

However the .gitlab-ci.yml provides support for only one image

This is not true. From the official documentation :

Your image will be named after the following scheme:

 <registry URL>/<namespace>/<project>/<image> 

GitLab supports up to three levels of image repository names. Following examples of image tags are valid:

 registry.example.com/group/project:some-tag registry.example.com/group/project/image:latest registry.example.com/group/project/my/image:rc1 

---

So the solution to your problem is simple - just build individual images and push them to GitLab container registry under different image name. The following config should give you an idea:

.template: &build_template
  image: docker:stable
  services:
    - docker:dind
  before_script:
    - docker login -u $CI_REGISTRY_USER -p $CI_REGISTRY_PASSWORD $CI_REGISTRY
  script:
    - docker pull $CI_REGISTRY_IMAGE/$IMAGE_NAME:latest || true
    - if [ -z ${CI_COMMIT_TAG+x} ];
      then docker build
        --cache-from $CI_REGISTRY_IMAGE/$IMAGE_NAME:latest
        --file $DOCKERFILE_NAME
        --tag $CI_REGISTRY_IMAGE/$IMAGE_NAME:$CI_COMMIT_SHA
        --tag $CI_REGISTRY_IMAGE/$IMAGE_NAME:$CI_COMMIT_TAG
        --tag $CI_REGISTRY_IMAGE/$IMAGE_NAME:latest . ;
      else docker build
        --cache-from $CI_REGISTRY_IMAGE/$IMAGE_NAME:latest
        --file $DOCKERFILE_NAME
        --tag $CI_REGISTRY_IMAGE/$IMAGE_NAME:$CI_COMMIT_SHA
        --tag $CI_REGISTRY_IMAGE/$IMAGE_NAME:latest . ;
      fi
    - docker push $CI_REGISTRY_IMAGE/$IMAGE_NAME:$CI_COMMIT_SHA
    - if [ -z ${CI_COMMIT_TAG+x} ]; then
        docker push $CI_REGISTRY_IMAGE/$IMAGE_NAME:$CI_COMMIT_TAG;
      fi
    - docker push $CI_REGISTRY_IMAGE/$IMAGE_NAME:latest

build:image1:
  <<: *build_template
  variables:
    IMAGE_NAME: image1
    DOCKERFILE_NAME: Dockerfile.1

build:image2:
  <<: *build_template
  variables:
    IMAGE_NAME: image2
    DOCKERFILE_NAME: Dockerfile.2

And you should be able to pull the same image using $CI_REGISTRY_IMAGE/$IMAGE_NAME:$CI_COMMIT_SHA in later pipeline jobs or your compose file (provided that the variables are passed to where you run your compose file).

Answer 4

In my case changing the url causes repository not exists error. So a workaround is to tag different images with different tag. For example I have a project of Drupal which contains two images: one for Drupal source code & second for MySQL database.

I tagged them:

docker build -t registry.mysite.net/drupal/blog:db mysql/db
docker build -t registry.mysite.net/drupal/blog:drupal src/drupal

Here you can see we are not changing the repo url: registry.mysite.net/drupal/blog:drupal

We are only changing the tags. And then to push it to gitlab like:

docker push registry.mysite.net/drupal/blog:db
docker push registry.mysite.net/drupal/blog:drupal

Hope this might help someone.