Run aarch64 Docker image in GitHub x86 container workflow

Question

I'm testing a GitHub workflow that checks if the code compiles on arm64, using a Linux/aarch64 Docker container for the build. I'm aware GitHub containers are x86 only, however I'd like to know if it is possible to execute multi-arch images in docker on GitHub managed workflows.

• I've built an image with 'docker buildx' set up for linux/amd64, linux/arm64 as per https://www.docker.com/blog/multi-arch-images/
• the packaged image manifest shows it is multi-arch and testing manually it works.

Details

• image https://github.com/donsbot/hsthrift/pkgs/container/hsthrift%2Fci-base/13306041?tag=ghcup-arm64v8
• workflow to boot this https://github.com/facebookincubator/hsthrift/actions/runs/1706096700/workflow

On GitHub the CI job is executed and the container initializes ok: https://github.com/facebookincubator/hsthrift/runs/4835956150?check_suite_focus=true#step:2:14

  /usr/bin/docker pull ghcr.io/donsbot/hsthrift/ci-base:ghcup-arm64v8
  ghcup-arm64v8: Pulling from donsbot/hsthrift/ci-base
...
  Digest: sha256:4c09341793d78efb74ad751b55152637d00b6297049458923825368fffb5485d
  Status: Downloaded newer image for ghcr.io/donsbot/hsthrift/ci-base:ghcup-arm64v8
  ghcr.io/donsbot/hsthrift/ci-base:ghcup-arm64v8

  /usr/bin/docker create .. --cpus 2 ... --entrypoint "tail" ghcr.io/donsbot/hsthrift/ci-base:ghcup-arm64v8 "-f" "/dev/null"
  a63c28c54f278781f85a4ca85e3b1f5c3dd7c0d6b125fbcfd0e1d2f57ad0b0ef

  /usr/bin/docker start a63c28c54f278781f85a4ca85e3b1f5c3dd7c0d6b125fbcfd0e1d2f57ad0b0ef
  a63c28c54f278781f85a4ca85e3b1f5c3dd7c0d6b125fbcfd0e1d2f57ad0b0ef

  /usr/bin/docker ps --all --filter id=a63c28c54f278781f85a4ca85e3b1f5c3dd7c0d6b125fbcfd0e1d2f57ad0b0ef --filter status=running --no-trunc --format "{{.ID}} {{.Status}}"
  a63c28c54f278781f85a4ca85e3b1f5c3dd7c0d6b125fbcfd0e1d2f57ad0b0ef Up Less than a second

  /usr/bin/docker inspect --format "{{range .Config.Env}}{{println .}}{{end}}" a63c28c54f278781f85a4ca85e3b1f5c3dd7c0d6b125fbcfd0e1d2f57ad0b0ef
  CI=true
  HOME=/github/home
  GITHUB_ACTIONS=true
  PATH=/root/.ghcup/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin
  TZ=Europe/London
  LD_LIBRARY_PATH=/usr/local/lib:/usr/lib/x86_64-linux-gnu:

So that all looks fine and happy. But the job fails on the first exec step as the VM is not running.

Error response from daemon: Container a63c28c54f278781f85a4ca85e3b1f5c3dd7c0d6b125fbcfd0e1d2f57ad0b0ef is not running

This works successfully locally on various hosts.

Question

• how can I debug what's happening to the container here?
• are multi-arch images generally supported on x86 Docker containers on GitHub? (other examples)

Related work

• https://github.community/t/ask-about-github-hosted-runners-arm64-platform-support/167483/3
• https://github.com/uraimo/run-on-arch-action

Answer 1

This missing piece is multi-arch support in the standard GitHub 'container' docker engine.

Instead of using the 'container' method, we can use invoke the cross-compile steps in a fresh docker container manually. This is automated by the run-on-arch-action, which takes care of getting qemu and docker into the right state.

docker run --rm --privileged multiarch/qemu-user-static --reset -p yes

https://github.com/uraimo/run-on-arch-action

The yml file for building on non-x86 now looks like:

 build-on-aarch64:
    runs-on: ubuntu-latest
    name: ci (arm64)
    steps:
      - name: Checkout code
        uses: actions/checkout@v2
      - name: Build in arm64 container
        uses: uraimo/run-on-arch-action@v2.1.1
        with:
          arch: aarch64
          distro: ubuntu20.04
          # Install deps into the container. With the token, the container will be cached
          # The image is cached publically like a package
          githubToken: ${{ github.token }}
          install: |
             .. install packages for the arm container (e.g. Dockerfile steps)
          run: |
             .. run build steps on the container

This action essentially builds the Docker image for us, with qemu set up, takes care of caching it, and runs it with the right emulation for the GitHub runner hosts.