在 gitlab ci 中查找在 docker-compose 中运行的容器的 url/ip

Question

我有一个在 docker-compose 中运行的应用程序（用于验收测试）。 验收测试在本地工作，但它们需要运行在 docker-compose 中的 webservice 容器的主机（或 ip）才能向其发送请求。 这在本地工作正常，但是当它在 gitlab ci 服务器中运行时我找不到容器的 ip。 我已经尝试了以下几个解决方案（所有这些在本地运行时都有效，但在 gitlab ci 中均无效）来查找在 gitlab ci 服务器中的 docker-compose 中运行的容器的 url：

使用“docker”作为主机。 这适用于在 docker 中运行的应用程序，但不适用于 docker-compose
使用 docker-inspect 查找容器的 ip ( docker inspect -f '{{range .NetworkSettings.Networks}}{{.IPAddress}}{{end}}' reading-comprehension )
使用 docker-compose.yml 中的网络为容器分配一个静态 ip（最新尝试）。

gitlab ci 文件可以在这里找到： https ://gitlab.com/connorbutch/reading-comprehension/-/blob/9-list-all-assessments/.gitlab-ci.yml

    image: connorbutch/gradle-and-java-11:alpha

variables:
  GRADLE_OPTS: "-Dorg.gradle.daemon=false"
  DOCKER_HOST: "tcp://docker:2375"
  DOCKER_DRIVER: "overlay2"

before_script:
  - export GRADLE_USER_HOME=`pwd`/.gradle

services:
  - docker:stable-dind

stages:
  - build
  - docker_build
  - acceptance_test

unit_test:
  stage: build
  script: ./gradlew check
  cache:
    key: "$CI_COMMIT_REF_NAME"
    policy: pull
    paths:
      - build
      - .gradle

build:
  stage: build
  script:
    - ./gradlew clean quarkusBuild
    - ./gradlew clean build -Dquarkus.package.type=native -Dquarkus.native.container-build=true
  cache:
    key: "$CI_COMMIT_REF_NAME"
    policy: push
    paths:
      - build
      - .gradle
  artifacts:
    paths:
      - reading-comprehension-server-quarkus-impl/build/

docker_build:
  stage: docker_build
  script:
    - cd reading-comprehension-server-quarkus-impl
    - docker build -f infrastructure/Dockerfile -t registry.gitlab.com/connorbutch/reading-comprehension:$CI_COMMIT_SHORT_SHA  .
    - docker login -u $CI_REGISTRY_USER -p $CI_REGISTRY_PASSWORD $CI_REGISTRY
    - docker push registry.gitlab.com/connorbutch/reading-comprehension:$CI_COMMIT_SHORT_SHA

acceptance_test:
  stage: acceptance_test
  only:
    - merge_requests
  script:
    - docker login -u $CI_REGISTRY_USER -p $CI_REGISTRY_PASSWORD $CI_REGISTRY
    - cd reading-comprehension-server-quarkus-impl/infrastructure
    - export IMAGE_TAG=$CI_COMMIT_SHORT_SHA
    - docker-compose up -d & ../../wait-for-it-2.sh
    - cd ../..
    - ./gradlew -DBASE_URL='192.168.0.8' acceptanceTest
  artifacts:
    paths:
      - reading-comprehension/reading-comprehension-server-quarkus-impl/build/

docker-compose 文件可以在这里找到： https : //gitlab.com/connorbutch/reading-comprehension/-/blob/9-list-all-assessments/reading-comprehension-server-quarkus-impl/infrastructure/docker-编写.yml

在此处查找失败作业之一的输出： https : //gitlab.com/connorbutch/reading-comprehension/-/jobs/734771859

#This file is NOT ever intended for use in production.  Docker-compose is a great tool for running
#database with our application for acceptance testing.
version: '3.3'

networks:
  network:
    ipam:
      driver: default
      config:
        - subnet: 192.168.0.0/24

services:
  db:
    image: mysql:5.7.10
    container_name: "db"
    restart: always
    environment:
       MYSQL_DATABASE: "rc"
       MYSQL_USER: "user"
       MYSQL_PASSWORD: "password"
       MYSQL_ROOT_PASSWORD: "password"
       MYSQL_ROOT_HOST: "%"
    networks:
      network:
        ipv4_address: 192.168.0.4
    ports:
      - '3306:3306'
    expose:
      - '3306'
    volumes:
      - db:/var/lib/mysql
  reading-comprehension-ws:
    image: "registry.gitlab.com/connorbutch/reading-comprehension:${IMAGE_TAG}"
    container_name: "reading-comprehension"
    restart: on-failure
    environment:
      WAIT_HOSTS: "db:3306"
      DB_USER: "user"
      DB_PASSWORD: "password"
      DB_JDBC_URL: "jdbc:mysql://192.168.0.4:3306/rc"
    networks:
      network:
        ipv4_address: 192.168.0.8
    ports:
      - 8080:8080
    expose:
      - 8080
volumes:
  db:

有没有人知道如何访问在 gitlab ci 服务器上的 docker-compose 中运行的容器的 ip？ 欢迎任何建议。

谢谢，

康纳

Answer 1

这有点棘手，就在几天前，我遇到了类似的问题，但是从 CI 到客户端的 VPN :)

编辑：本地 gitlab 实例的解决方案

为 gitlab 跑步者创建自定义网络：

docker network create --subnet=172.16.0.0/28 \
 --opt com.docker.network.bridge.name=gitlab-runners \
 --opt com.docker.network.bridge.enable_icc=true \
 --opt com.docker.network.bridge.enable_ip_masquerade=true \
 --opt com.docker.network.bridge.host_binding_ipv4=0.0.0.0 \
 --opt com.docker.network.driver.mtu=9001 gitlab-runners

将新网络附加到 gitlab-runners

# /etc/gitlab-runner/config.toml
[[runners]]
....
   [runners.docker]
   ....
   network_mode = "gitlab-runners"

重新启动跑步者。

最后是 gitlab-ci.yml

start-vpn:
    stage: prepare-deploy
    image: docker:stable
    cache: {}
    variables:
        GIT_STRATEGY: none
    script:
        - >
            docker run -it -d --rm
            --name vpn-branch-$CI_COMMIT_REF_NAME
            --privileged
            --net gitlab-runners
            -e VPNADDR=$VPN_SERVER
            -e VPNUSER=$VPN_USER
            -e VPNPASS=$VPN_PASSWORD
            auchandirect/forticlient || true && sleep 2
        - > 
            docker inspect -f '{{range .NetworkSettings.Networks}}{{.IPAddress}}{{end}}' 
            vpn-branch-$CI_COMMIT_REF_NAME > vpn_container_ip
    artifacts:
        paths:
        - vpn_container_ip

在下一步中，您可以使用以下内容：

before_script:
        - ip route add 10.230.163.0/24 via $(cat vpn_container_ip) # prod/dev
        - ip route add 10.230.164.0/24 via $(cat vpn_container_ip) # test

编辑：gitlab.com 的解决方案

基于gitlab 问题， DinD 中的答案端口映射与 nonDinD gitlab-runner 略有不同，对于暴露的端口，您应该使用主机名“docker”。

例子：

services:
  - docker:stable-dind

variables:
  DOCKER_HOST: "tcp://docker:2375"

stages:
  - test

test env:
  image: tmaier/docker-compose:latest
  stage: test
  script:
    # containous/whoami with exposed port 80:80
    - docker-compose up -d
    - apk --no-cache add curl
    - curl docker:80              # <------- 
    - docker-compose down

Answer 2

我正在使用 docker 而不是 docker-compose 并且上面的解决方案对我不起作用/我正在使用基于节点的我自己的图像，我在其中安装了 docker & buildx 像这样：

ARG NODE_VER=lts-alpine

FROM node:${NODE_VER}

ARG BUILDX_VERSION=0.5.1
ARG DOCKER_VERSION=20.10.6
ARG BUILDX_ARCH=linux-arm-v7

RUN apk --no-cache add curl

# install docker
RUN curl -SL "https://download.docker.com/linux/static/stable/armhf/docker-${DOCKER_VERSION}.tgz" | \
        tar -xz --strip-components 1 --directory /usr/local/bin/
COPY docker/modprobe.sh /usr/local/bin/modprobe
# replace node entrypoint by docker one /!\
COPY docker/docker-entrypoint.sh /usr/local/bin/
ENV DOCKER_TLS_CERTDIR=/certs
RUN mkdir /certs /certs/client && chmod 1777 /certs /certs/client

# download buildx
RUN mkdir -p /usr/lib/docker/cli-plugins \
 && curl -L \
      --output /usr/lib/docker/cli-plugins/docker-buildx \
      "https://github.com/docker/buildx/releases/download/v${BUILDX_VERSION}/buildx-v${BUILDX_VERSION}.${BUILDX_ARCH}"
RUN chmod a+x /usr/lib/docker/cli-plugins/docker-buildx
RUN mkdir -p /etc/docker && echo '{"experimental": true}' > /usr/lib/docker/config.json

我的 gitlab-ci.yml 包含：

image: myimageabove

variables:
  DOCKER_DRIVER: overlay2
  PLATFORMS: linux/arm/v7
  IMAGE_NAME: ${CI_PROJECT_NAME}
  TAG: ${CI_COMMIT_BRANCH}-latest
  REGISTRY: registry.gitlab.com
  REGISTRY_ROOT: mygroup
  WEBSOCKETD_VER: 0.4.1
#  DOCKER_GATEWAY_HOST: 172.17.0.1
  DOCKER_GATEWAY_HOST: docker

services:
  - docker:dind

before_script:
  - docker info

build:
  stage: build
  script:
    - docker login -u "$CI_REGISTRY_USER" -p "$CI_REGISTRY_PASSWORD" ${REGISTRY}
    - docker buildx create --use
    - docker buildx build --platform $PLATFORMS --tag "${REGISTRY}/${REGISTRY_ROOT}/${IMAGE_NAME}:${TAG}" --push .

test:
  stage: test
  variables:
    WSD_DIR: /tmp/websocketd
    WSD_FILE: /tmp/websocketd/websocketd
  cache:
    key: websocketd
    paths: 
      - ${WSD_DIR}
  before_script:
# download websocketd and put in cache if needed
    - if [ ! -f ${WSD_FILE} ]; then
        mkdir -p ${WSD_DIR};
        curl -o ${WSD_FILE}.zip -L "https://github.com/joewalnes/websocketd/releases/download/v${WEBSOCKETD_VER}/websocketd-${WEBSOCKETD_VER}-linux_arm.zip";
        unzip -o ${WSD_FILE}.zip websocketd -d ${WSD_DIR};
        chmod 755 ${WSD_FILE};
      fi;
    - mkdir /home/pier
    - cp -R ./test/resources/* /home/pier
    # get websocketd from cache
    - cp ${WSD_FILE} /home/pier/Admin/websocketd
    # setup envt variables
    - JWT_KEY=$(cat /home/pier/Services/Secrets/WEBSOCKETD_KEY)
    # - DOCKER_GATEWAY_HOST=$(/sbin/ip route|awk '/default/ { print $3 }')
    # - DOCKER_GATEWAY_HOST=$(hostname)
    - ENVT="-e BASE_URL=/ -e JWT_KEY=$JWT_KEY -e WEBSOCKETD_KEY=$JWT_KEY -e WEBSOCKET_URL=ws://${DOCKER_GATEWAY_HOST:-host.docker.internal}:8088 -e SERVICES_DIR=/home/pier/Services"
    - VOLUMES='-v /tmp:/config -v /home/pier/Services:/services -v /etc/wireguard:/etc/wireguard'

  script:
    # start websocketd
    - /home/pier/start.sh &
    # start docker pier admin
    - docker run -p 4000:4000 ${ENVT} ${VOLUMES} ${REGISTRY}/${REGISTRY_ROOT}/${IMAGE_NAME}:${TAG}
    # run postman tests
    - newman run https://api.getpostman.com/collections/${POSTMAN_COLLECTION_UID}?apikey=${POSTMAN_API_KEY}

deploy:
  stage: deploy
  script:
    # just push to docker hub
    - docker login -u "$DOCKERHUB_REGISTRY_USER" -p "$DOCKERHUB_REGISTRY_PASSWORD" ${DOCKERHUB}
    - docker buildx build --platform $PLATFORMS --tag "${DOCKERHUB}/mygroup/${IMAGE}:${TAG}" --push .

当我运行它时，构建工作正常工作，然后测试“before_script”工作但是当脚本启动时，我得到以下跟踪：

# <= this starts the websocketd server locally on port 8088 =>
$ /home/pier/start.sh &
# <= this starts the image I just built which should connect to the above websocketd server =>
$ docker run -p 4000:4000 ${ENVT} ${VOLUMES} ${REGISTRY}/${REGISTRY_ROOT}/${IMAGE_NAME}:${TAG}     
# <= trace of the websocketd server start with url ws://runner-hbghjvzp-project-22314059-concurrent-0:8088/ =>
Tue, 11 May 2021 12:08:13 +0000 | INFO   | server     |  | Serving using application   : ./websocket-server.py 
Tue, 11 May 2021 12:08:13 +0000 | INFO   | server     |  | Starting WebSocket server   : ws://runner-hbghjvzp-project-22314059-concurrent-0:8088/
# <= trace of the image start saying it tires to conenct to the websocketd server
Websocket connecting to ws://docker:8088 ...
Listen on port 4000
# <= trace with ENOTFOUND on "docker" address =>
websocket connection failed: Error: getaddrinfo ENOTFOUND docker
    at GetAddrInfoReqWrap.onlookup [as oncomplete] (dns.js:67:26) {
  errno: -3008,
  code: 'ENOTFOUND',
  syscall: 'getaddrinfo',
  hostname: 'docker'
}
/pier/storage/websocket-client.js:52
            throw err;
            ^
Error: getaddrinfo ENOTFOUND docker
    at GetAddrInfoReqWrap.onlookup [as oncomplete] (dns.js:67:26) {
  errno: -3008,
  code: 'ENOTFOUND',
  syscall: 'getaddrinfo',
  hostname: 'docker'
}
Cleaning up file based variables 00:01
ERROR: Job failed: exit code 1

我尝试了其他方法，例如：

websocket connection failed: Error: getaddrinfo ENOTFOUND host.docker.internal
websocket connection failed: Error: connect ETIMEDOUT 172.17.0.1:8088 # <= same error when trying $(/sbin/ip route|awk '/default/ { print $3 }') =>
websocket connection failed: Error: getaddrinfo ENOTFOUND runner-meuessxe-project-22314059-concurrent-0  # using $(hostname)

出于新的想法......将不胜感激任何帮助。

在 gitlab ci 中查找在 docker-compose 中运行的容器的 url/ip

问题描述

2 个解决方案

解决方案1
1 2020-09-15 14:53:18

编辑：本地 gitlab 实例的解决方案

编辑：gitlab.com 的解决方案

解决方案2
0 2021-05-11 14:45:36

在 gitlab ci 中查找在 docker-compose 中运行的容器的 url/ip

问题描述

2 个解决方案

解决方案1 1 2020-09-15 14:53:18

编辑：本地 gitlab 实例的解决方案

编辑：gitlab.com 的解决方案

解决方案2 0 2021-05-11 14:45:36

解决方案1
1 2020-09-15 14:53:18

解决方案2
0 2021-05-11 14:45:36