Communicate between two containers in Google cloud build

Question

I am running my CI/CD pipeline in Google cloud build. My app has web and wget containers. I am trying to reach web from wget

Cloud build internally used cloudbuild bridge network while starting containers as steps. So I am expecting these steps to communicate using names. But its failing.

If I create my own docker bridge netwok then they communicating.

I want to know why cloudbuild network is not working as expected.

Please let me know if you know any other ways to establish communication between step containers.

cloudbuild.yaml

steps:

- name: 'gcr.io/cloud-builders/docker'
  id: Web server
  args: ["run", "-d", "--name", "mani", "manikantanr/hostname_ip"]

- name: 'gcr.io/cloud-builders/wget'
  id: wget web mani:8000
  args: ["-qO-", "http://mani:8000"]

To understand the cloudbuild internals I used few docker commands.

debug-cloudbuild.yaml

steps:

- name: 'gcr.io/cloud-builders/docker'
  id: Docker Version
  args: ["version"]

- name: 'gcr.io/cloud-builders/docker'
  id: Docker info
  args: ["info"]

- name: 'gcr.io/cloud-builders/docker'
  id: Docker volume ls
  args: ["volume", "ls"]

- name: 'gcr.io/cloud-builders/docker'
  id: Docker volume inspect homevol
  args: ["volume", "inspect", "homevol"]


- name: 'gcr.io/cloud-builders/docker'
  id: Docker network ls
  args: ["network", "ls"]

- name: 'gcr.io/cloud-builders/docker'
  id: Docker network inspect cloudbuild
  args: ["network", "inspect", "cloudbuild"]

- name: 'gcr.io/cloud-builders/docker'
  id: Docker ps before
  args: ["container", "ls", "--no-trunc"]

- name: 'gcr.io/cloud-builders/docker'
  id: Web server
  args: ["run", "-d", "--name", "mani", "manikantanr/hostname_ip"]
  # waitFor: ['-']

- name: 'gcr.io/cloud-builders/wget'
  id: wget ipinfo
  args: ["-qO-", "https://ipinfo.io"]

- name: 'gcr.io/cloud-builders/docker'
  id: Docker ps after
  args: ["container", "ls", "--no-trunc"]

- name: 'gcr.io/cloud-builders/docker'
  id: Docker inspect mani host network
  args: ["inspect", "mani"]

- name: 'gcr.io/cloud-builders/docker'
  id: Docker alpine ifconfig inside container
  args: ["run", "alpine", "ifconfig"]

- name: 'gcr.io/cloud-builders/wget'
  id: wget mani:8000
  args: ["-qO-", "http://mani:8000"]

Answer 1

I had a similar issue setting up integration tests on cloud build. I was trying to run integration tests from another builder (go-builder) against my other containers (started through docker-compose community built containers).

Without specifying any networks on docker-compose.yaml, all containers are started on the default network ( https://docs.docker.com/compose/networking/ ). On cloud build, it creates a new network named cloudbuild_default and places all my containers there. By forcing all containers to join cloudbuild network through my docker-compose.yaml file, I was able to establish communications and run my tests against them.

#docker-compose.yaml

networks:
  default:
    external:
      name: cloudbuild

This might be an alternate configuration for you. Hope it helps

Answer 2

From the docs :

Each build step is run with its container attached to a local Docker network named cloudbuild. This allows build steps to communicate with each other and share data.

You can use docker compose and using cloudbuild network, for example:

#docker-compose.yml
app-workspace:
  ...
  network_mode: cloudbuild
db-mysql:
  ...
  network_mode: cloudbuild
...
networks:
  default:
    external:
      name: cloudbuild

Or if you are using docker run , add option --network cloudbuild .

After that, you can communicate to other services you defined in the previous step as you expect. For example:

#steps
- id: 'Ping to other container'
  name: gcr.io/cloud-builders/curl
  args: ["app-workspace:your-service-port"]

Hope this helps.

Answer 3

I did an experiment and it looks like (without doing any special setup) you can communicate between build step containers by using the name step_x (0-based numbering).

For example if you have a web-server listening on the endpoint /hello (on port 8081) in the container for the first build step (step_0). You can make requests to that endpoint from another build step container by making a request to http://step_0:8081/hello .