How to setup multibranch CI with jenkins and docker-compose

Question

I have configured CI pipeline with Jenkins for app stored in git and deployed with docker-compose. The jenkins agent used is a docker container which can run other containers including the docker-compose deployment.

To resume the jenkins CI:

• push on a specific branch of the git project
• jenkins multibranch job is triggered automatically
• build docker images of app
• run docker-compose to start app
• run tests of app
• stop docker-compose

How tests works:

• In the docker-compose, there is specific container that run the python tests.
• He can comunicate with other containers by container_name and because all containers are in the same docker.network

Problems:

• We want to run tests every push on git and on every branches
• When two developers works on the app but on two different branches, jenkins triggers 2 jobs in multi-branch job for each branches. The problem is, the two jenkins jobs try to deploy the same docker-compose at the same time. So he can't because of container name conflict

Questions:

• I want to know what is the best practices for CI when you want to run tests after a push on different branches in a git project.
• because with this method, the risk of broken job is very high.

Answer 1

I want to know what is the best practices for CI when you want to run tests after a push on different branches in a git project.

Use docker in docker and run your docker-compose inside a dockerd running in your custom docker-container spawned for the specific CI.

https://www.jenkins.io/doc/book/pipeline/docker/

https://hub.docker.com/layers/docker/library/docker/dind/images/sha256-34c594c12507d24f66789978a90240d4addf3eb8fce22a7a9157ca9701afd46d?context=explore

https://docs.gitlab.com/ee/ci/docker/using_docker_build.html