docker build 在 Ubuntu 16.04 上运行命令失败，但在 18.04 上使用相同的 Docker 版本失败

Question

I'm running the following Dockerfile successfully on my development machine with Ubuntu 18.04.01:

FROM arm32v7/openjdk:8-jdk-slim

WORKDIR /app

COPY qemu-arm-static /usr/bin/qemu-arm-static

COPY ./target/edge-0.0.1-SNAPSHOT.jar ./app.jar
COPY ./config/ ./config

RUN groupadd --gid 1337 moduleuser && \
    useradd --uid 1337 --gid moduleuser --shell /bin/bash --create-home moduleuser

RUN mkdir -p /var/opt
RUN chown moduleuser:moduleuser -R /var/opt

USER moduleuser
ENTRYPOINT ["java","-jar","./app.jar"]

However, when I try to build it in our CI environment, all nodes have 16.04, and I can't seem to run it in those environments. The output of the command run at 16.04 is as follows:

$ sudo docker build -f Dockerfile.arm32v7 .
Sending build context to Docker daemon  32.87MB
Step 1/10 : FROM arm32v7/openjdk:8-jdk-slim
 ---> e5f4973cadb1
Step 2/10 : WORKDIR /app
 ---> Using cache
 ---> 44da7413978b
Step 3/10 : COPY qemu-arm-static /usr/bin/qemu-arm-static
 ---> Using cache
 ---> ca8c3bde0d92
Step 4/10 : COPY ./target/edge-0.0.1-SNAPSHOT.jar ./app.jar
 ---> Using cache
 ---> 90267cfe2fe1
Step 5/10 : COPY ./config/ ./config
 ---> Using cache
 ---> 1612aca9fa90
Step 6/10 : RUN groupadd --gid 1337 moduleuser &&     useradd --uid 1337 --gid moduleuser --shell /bin/bash --create-home moduleuser
 ---> Running in 7a29d218f15e
standard_init_linux.go:207: exec user process caused "exec format error"
The command '/bin/sh -c groupadd --gid 1337 moduleuser &&     useradd --uid 1337 --gid moduleuser --shell /bin/bash --create-home moduleuser' returned a non-zero code: 1

For reference, here is the output from the successful build at 18.04:

$ docker build -f ./Dockerfile.arm32v7 .
Sending build context to Docker daemon  32.85MB
Step 1/10 : FROM arm32v7/openjdk:8-jdk-slim
 ---> e5f4973cadb1
Step 2/10 : WORKDIR /app
 ---> Using cache
 ---> b8dc45ea966c
Step 3/10 : COPY qemu-arm-static /usr/bin/qemu-arm-static
 ---> Using cache
 ---> 43f2b39b8455
Step 4/10 : COPY ./target/edge-0.0.1-SNAPSHOT.jar ./app.jar
 ---> Using cache
 ---> bce9640496a9
Step 5/10 : COPY ./config/ ./config
 ---> Using cache
 ---> 70c4f5ab3cd1
Step 6/10 : RUN groupadd --gid 1337 moduleuser &&     useradd --uid 1337 --gid moduleuser --shell /bin/bash --create-home moduleuser
 ---> Using cache
 ---> 862b0e48f546
Step 7/10 : RUN mkdir -p /var/opt
 ---> Using cache
 ---> 55545e275209
Step 8/10 : RUN chown moduleuser:moduleuser -R /var/opt
 ---> Using cache
 ---> 9a5e061b4b84
Step 9/10 : USER moduleuser
 ---> Using cache
 ---> 8b049ddef6a6
Step 10/10 : ENTRYPOINT ["java","-jar","./app.jar"]
 ---> Using cache
 ---> 433e7e20be3a
Successfully built 433e7e20be3a

If I start a an interactive session on the 16.04 machine with the image that arm32v7/openjdk:8-jdk-slim is based on using sudo docker run -it debian:stretch-slim , I can succesfully run the commands individually like this:

root@557efbe64410:/# groupadd --gid 1337 moduleuser
root@557efbe64410:/# useradd --uid 1337 --gid moduleuser --shell /bin/bash --create-home moduleuser

But when I run the command that causes the error from the build, I instead the manual from the useradd command:

Both machines are running Docker version 18.09.1, build 4c52b90.

I'm at a loss here, I can't figure out what is going wrong. Any help would be greatly appreciated.

Additional information from questions in comments:

16.04:

$ docker system info --format '{{.Architecture}}'
x86_64
$ uname -m
x86_64

18.04:

$ docker system info --format '{{.Architecture}}'
x86_64
$ uname -m
x86_64

The 18.04 machine has lots of other qemu binaries installed. I added a gist with themhere . However, I was under the impression qemu-arm-static could run standalone and didn't depend on any other binaries.

I am not using binfmt utils, I am simply emulating ARM using qemu inside the image.

Answer 1

It turns out qemu-arm-static is not as portable as I first thought. When you install qemu-user-static , a mapping is added to the kernel to make it understand that it should use qemu to interpret that architecture, exactly like a comment said above. I didn't know this is what happened under the hood.

So there are two alternatives to resolve this:

sudo apt install qemu-user-static

or

mount binfmt_misc -t binfmt_misc /proc/sys/fs/binfmt_misc  
echo ':arm:M::\x7fELF\x01\x01\x01\x00\x00\x00\x00\x00\x00\x00\x00\x00\x02\x00\x28\x00:\xff\xff\xff\xff\xff\xff\xff\x00\xff\xff\xff\xff\xff\xff\xff\xff\xfe\xff\xff\xff:/usr/bin/qemu-arm-static:' > /proc/sys/fs/binfmt_misc/register  

Source: https://www.balena.io/blog/building-arm-containers-on-any-x86-machine-even-dockerhub/