OCI 运行时创建失败：container_linux.go:349：启动容器进程导致“exec:\\”xxxx\\“：在 $PATH 中找不到可执行文件”：未知

Question

I am using CentOS 7.4.1708. Current environment have no internet access. So I installed docker (and docker-compose) from binaries.

At first, I encounter below error, found that is related to selinux with docker. So I disable selinux.

[root@DB1 mysql-docker]# docker-compose up -d 
Starting mysql-docker_db_1 ... 
INFO[2020-08-27T10:18:39.817597300+08:00] shim containerd-shim started                  address="/containerd-shim/moby/596c3ab06926e99ffeccca8ffebfa08a540ca792517285f7a9cdc4855508af38/shim.sock" debug=false pid=5110
INFO[2020-08-27T10:18:39.896480000+08:00] shim reaped                                   id=596c3ab06926e99ffeccca8ffebfa08a540ca792517285f7a9cdc4855508af38
ERRO[2020-08-27T10:18:39.916766100+08:00] stream copy error: reading from a closed fifo 
ERRO[2020-08-27T10:18:39.928696800+08:00] stream copy error: reading from a closed fifo 
ERRO[2020-08-27T10:18:40.077880000+08:00] 596c3ab06926e99ffeccca8ffebfa08a540ca792517285f7a9cdc4855508af38 cleanup: failed to delete container from containerd: no such contaiStarting mysql-docker_db_1 ... error

ERROR: for mysql-docker_db_1  Cannot start service db: OCI runtime create failed: container_linux.go:349: starting container process caused "process_linux.go:449: container init caused \"write /proc/self/attr/keycreate: permission denied\"": unknown

ERROR: for db  Cannot start service db: OCI runtime create failed: container_linux.go:349: starting container process caused "process_linux.go:449: container init caused \"write /proc/self/attr/keycreate: permission denied\"": unknown
ERROR: Encountered errors while bringing up the project.

After disable selinux, I still got below error

[root@DB1 mysql-docker]# docker-compose up -d 
Removing mysql-docker_db_1
Recreating 596c3ab06926_mysql-docker_db_1 ... 
INFO[2020-08-27T11:06:24.808910400+08:00] shim containerd-shim started                  address="/containerd-shim/moby/3324ac0a68749295f631247fdd071d687fbeeb946920bc788af782e880797fc6/shim.sock" debug=false pid=5888
INFO[2020-08-27T11:06:25.178909600+08:00] shim reaped                                   id=3324ac0a68749295f631247fdd071d687fbeeb946920bc788af782e880797fc6
ERRO[2020-08-27T11:06:25.200213100+08:00] stream copy error: reading from a closed fifo 
ERRO[2020-08-27T11:06:25.200224900+08:00] stream copy error: reading from a closed fifo 
ERRO[2020-08-27T11:06:25.354324500+08:00] 3324ac0a68749295f631247fdd071d687fbeeb946920bc788af782e880797fc6 cleanup: failed to delete container from containerd: no such contaRecreating 596c3ab06926_mysql-docker_db_1 ... error

ERROR: for 596c3ab06926_mysql-docker_db_1  Cannot start service db: OCI runtime create failed: container_linux.go:349: starting container process caused "exec: \"--default-authentication-plugin=mysql_native_password\": executable file not found in $PATH": unknown

ERROR: for db  Cannot start service db: OCI runtime create failed: container_linux.go:349: starting container process caused "exec: \"--default-authentication-plugin=mysql_native_password\": executable file not found in $PATH": unknown
ERROR: Encountered errors while bringing up the project.

My docker-compose.yaml :

version: '3.8'

services:
  db:
    image: mysql:5.7
    container_name: db1_mysql
    command: --default-authentication-plugin=mysql_native_password
    restart: always
    environment:
      MYSQL_ROOT_PASSWORD: password
      MYSQL_USER: user
      MYSQL_PASS: password
    ports:
      - 3306:3306
    volumes:
      - ./db_data:/var/lib/mysql

I have tested on my local pc. It can start and connect to mysql.

Edit:

Got the same error when trying to run a centos image:

[user@host ~]$ docker run -it centos:7 /usr/bin/bash
INFO[2020-08-27T14:09:31.576862000+08:00] shim containerd-shim started                  address="/containerd-shim/moby/9c51066015093177fa65305ba3a8efe510f27ee6849f043ce73e82e71195680c/shim.sock" debug=false pid=2495
INFO[2020-08-27T14:09:31.702726300+08:00] shim reaped                                   id=9c51066015093177fa65305ba3a8efe510f27ee6849f043ce73e82e71195680c
ERRO[2020-08-27T14:09:31.723296400+08:00] stream copy error: reading from a closed fifo
ERRO[2020-08-27T14:09:31.752329700+08:00] 9c51066015093177fa65305ba3a8efe510f27ee6849f043ce73e82e71195680c cleanup: failed to delete container from containerd: no such container 
docker: Error response from daemon: OCI runtime create failed: container_linux.go:349: starting container process caused "exec: \"/usr/bin/bash\": stat /usr/bin/bash: no such file or directory": unknown.

docker image inspect mysql:5.7 :

[
    {
        "Id": "sha256:f40b94ddfb35bbb991c05191ffddf988a6c660c039f441a133ed35df5a71a8f0",
        "RepoTags": [
            "mysql:5.7"
        ],
        "RepoDigests": [],
        "Parent": "",
        "Comment": "Imported from -",
        "Created": "2020-08-27T04:00:13.8522196Z",
        "Container": "",
        "ContainerConfig": {
            "Hostname": "",
            "Domainname": "",
            "User": "",
            "AttachStdin": false,
            "AttachStdout": false,
            "AttachStderr": false,
            "Tty": false,
            "OpenStdin": false,
            "StdinOnce": false,
            "Env": null,
            "Cmd": null,
            "Image": "",
            "Volumes": null,
            "WorkingDir": "",
            "Entrypoint": null,
            "OnBuild": null,
            "Labels": null
        },
        "DockerVersion": "19.03.9",
        "Author": "",
        "Config": {
            "Hostname": "",
            "Domainname": "",
            "User": "",
            "AttachStdin": false,
            "AttachStdout": false,
            "AttachStderr": false,
            "Tty": false,
            "OpenStdin": false,
            "StdinOnce": false,
            "Env": null,
            "Cmd": null,
            "Image": "",
            "Volumes": null,
            "WorkingDir": "",
            "Entrypoint": null,
            "OnBuild": null,
            "Labels": null
        },
        "Architecture": "amd64",
        "Os": "linux",
        "Size": 453807757,
        "VirtualSize": 453807757,
        "GraphDriver": {
            "Data": {
                "MergedDir": "/var/lib/docker/overlay2/a6d62e03185b9cb1baf2e8a59f49c8e5b8a11ddbe6c724dbbeecf8e7d270f705/merged",
                "UpperDir": "/var/lib/docker/overlay2/a6d62e03185b9cb1baf2e8a59f49c8e5b8a11ddbe6c724dbbeecf8e7d270f705/diff",
                "WorkDir": "/var/lib/docker/overlay2/a6d62e03185b9cb1baf2e8a59f49c8e5b8a11ddbe6c724dbbeecf8e7d270f705/work"
            },
            "Name": "overlay2"
        },
        "RootFS": {
            "Type": "layers",
            "Layers": [
                "sha256:926d1cc1d1ceda07eca8dc3df0aa1da131a02cad0cf2ae52c28e7e4ee0eee69b"
            ]
        },
        "Metadata": {
            "LastTagTime": "2020-08-27T12:00:13.8580127+08:00"
        }
    }
]

Answer 1

The first image issue could be because you imported the image incorrectly. The docker import command is the inverse of docker export which works on container filesystems rather than images with image metadata. Instead you should run docker load with docker save to transfer images. If that works correctly, you will see an entrypoint defined in your image and that entrypoint is what docker should try running with the command value as an argument to the entrypoint:

$ docker image inspect mysql:5.7 --format '{{.Config.Entrypoint}}'
[docker-entrypoint.sh]

The second error looks like a bad export/import of the centos:7 image. If you are using overlay2 you can inspect the image and filesystem layers:

# docker image inspect centos:7 --format '{{.RootFS.Layers}}'
[sha256:77b174a6a187b610e4699546bd973a8d1e77663796e3724318a2a4b24cb07ea0]

# cat /var/lib/docker/image/overlay2/layerdb/sha256/77b174a6a187b610e4699546bd973a8d1e7
7663796e3724318a2a4b24cb07ea0/cache-id
e82a8ede7fba48074c4c41c53db8244002cb6896f0687e1af29d15a411de11c7

# ls -al /var/lib/docker/overlay2/e82a8ede7fba48074c4c41c53db8244002cb6896f0687e1af29d15a411de11c7/
committed  diff/      link
root@bmitch-t490:/home/bmitch# ls -al /var/lib/docker/overlay2/e82a8ede7fba48074c4c41c53db8244002cb6896f0687e1af29d15a411de11c7/diff/usr/bin/bash
-rwxr-xr-x 1 root root 964600 Aug  8  2019 /var/lib/docker/overlay2/e82a8ede7fba48074c4c41c53db8244002cb6896f0687e1af29d15a411de11c7/diff/usr/bin/bash

Answer 2

double-check the exact command you're running to start your container; from that output, it looks like you passed the -d option after the name of the image you're trying to run. Because of that, -d is used as the command to run inside the container, thus will fail;

The -d (--detach) option is an option for docker run, so must be passed before the name of the image you're running;

docker run -d containername