How to use docker buildx bake to replace docker-compose up?

Question

So I have this docker command that works fine with other machines but since mine is an M1 chip, I am having issues. I would like to use buildx and bake to be able to run the docker containers for multiple architectures. This is the command,

PY_VER=xxxx IMAGE=xxxx DISTRO=xxxxx \
PHARUS_VERSION=$(cat xxxxx/xxxx/xxxxx | tail -1 | awk -F\' '{print $2}') \
DJLABBOOK_VERSION=$(cat package.json | grep \"version\" | awk -F\" '{print $4}') \
HOST_UID=$(id -u) \
docker-compose -f docker-compose-dev.yaml up

I have a Dockerfile and everything set up but this isn't working with my M1 machine.

Whenever I run this command, docker buildx bake -f env.hcl app -f docker-bake.hcl I get this output.

 => [internal] load build definition from Dockerfile                                                                            0.1s
 => => transferring dockerfile: 32B                                                                                             0.1s
 => [internal] load .dockerignore                                                                                               0.0s
 => => transferring context: 34B                                                                                                0.0s
 => [internal] load metadata for docker.io/library/node:lts-buster-slim                                                         0.4s
 => [internal] load metadata for docker.io/library/golang:alpine3.11                                                            0.4s
 => [stage-1  1/11] FROM docker.io/library/node:lts-buster-slim@sha256:xxxxxxxxxxxxxxxxxxxxxxxxxxxxx 0.0s
 => [internal] load build context                                                                                               0.0s
 => => transferring context: 4.47kB                                                                                             0.0s
 => [go_tmp 1/3] FROM docker.io/library/golang:alpine3.11@sha256:xxxxxxxxxxxxxxx  0.0s
 => https://raw.githubusercontent.com/datajoint/miniconda3-docker/master/utilities/startup.go                                   0.1s
 => CACHED [stage-1  2/11] RUN apt-get update && apt-get install wget -y                                                        0.0s
 => CACHED [stage-1  3/11] WORKDIR /home/node                                                                                   0.0s
 => CACHED [stage-1  4/11] COPY --chown=node:node ./package.json ./package-lock.json  /home/node/                               0.0s
 => CACHED [stage-1  5/11] RUN npm install                                                                                      0.0s
 => CACHED [go_tmp 2/3] ADD https://raw.githubusercontent.com/datajoint/miniconda3-docker/master/utilities/startup.go /startup  0.0s
 => CACHED [go_tmp 3/3] RUN cd / && go build startup.go                                                                         0.0s
 => CACHED [stage-1  6/11] COPY --from=go_tmp /startup /startup                                                                 0.0s
 => CACHED [stage-1  7/11] COPY ./docker-entrypoint.sh /docker-entrypoint.sh                                                    0.0s
 => CACHED [stage-1  8/11] RUN   chmod +x /docker-entrypoint.sh &&   chmod 4755 /startup                                        0.0s
 => CACHED [stage-1  9/11] COPY --chown=node:node ./tsconfig.json /home/node/                                                   0.0s
 => CACHED [stage-1 10/11] COPY --chown=node:node ./public  /home/node/public                                                   0.0s
 => CACHED [stage-1 11/11] COPY --chown=node:node ./src  /home/node/src                                                         0.0s
 => exporting to image                                                                                                          0.0s
 => => exporting layers                                                                                                         0.0s
 => => writing image sha256:xxxxxxxxxxxxxxxxx                                    0.0s

This is the contents of my env.hcl file

PY_VER="3.8"

IMAGE="djbase"

DISTRO="alpine"

PHARUS_VERSION="$(cat pharus/pharus/version.py | tail -1 | awk -F\" '{print $2}')"

DJLABBOOK_VERSION="$(cat package.json | grep \"version\" | awk -F\" '{print $4}')"

HOST_UID="$(id -u)"

and the contents of my docker-bake.hcl

variable "PY_VER" {
    default = "xxx"
}

variable "IMAGE" {
    default = "xxx"
}

variable "DISTRO" {
    default = "xxxxx"
}

variable "PHARUS_VERSION" {
    default = "$(cat xxxx/xxxx/xxxxn.py | tail -1 | awk -F\" '{print $2}')"
}

variable "DJLABBOOK_VERSION" {
    default = "$(cat package.json | grep \"version\" | awk -F\" '{print $4}')"
}

variable "HOST_UID" {
    default = "$(id -u)"
}

target "app" {
    args = {
        PY_VER = "${PY_VER}"
        HOST_UID = "${HOST_UID}"
    }
}

Answer 1

buildx bake isn't meant to replace compose up , it's for building docker images. So it's more comparable to docker compose build or just docker build . Where you want to use buildx is in order to build images for multiple architectures.

The problem you're having with your M1 chip is likely that your docker image isn't meant to run on that architecture ( linux/arm64 ) because it's probably been built for x86 ( linux/amd64 ).

Make sure you have Rosetta 2 installed, and your system should be able to emulate x86 and run your images anyway. Check the docs here for more .

Running emulated images under docker is slow though. So what I've been doing is using bulidx to build images for multiple architectures then you can pull those images with docker composer.

# docker-bake.hcl
variable "platforms" {
  default = ["linux/amd64", "linux/arm64"]
}

group "default" {
  targets = [
    "my_image",
  ]
}

target "my_image" {
  dockerfile = "myimage.Dockerfile"
  tags = ["myrepo/myimage:latest"]
  platforms = platforms
}

# Command
docker build bake --push

# docker-compose.yml
version: '3.9'
services:
    dev:
        image: myrepo/myimage:latest

Now you'll have a docker image pushed for both architectures and docker will automatically select the correct one depending on the system that it's running on.

You'll probably have to configure buildx, check the docker multi-arch docs for more.

Answer 2

1. Build your images by using buildx and bake .
2. Put --load argument or --push respectively following your case.

If you use single architecture, just docker-compose down and docker-compose up --force-recreate or whatever command you can use the latest built image.

In the case of multi-platforms, you must pull the docker image from the remote repository and do compose down & up.