Docker 容器不健康但日志中没有错误

Question

I am using the official docker-compose file of airflow to spin it up.

Some of my containers seem unhealthy:

34d8698d67e7   apache/airflow:2.0.2   "/usr/bin/dumb-init …"   31 minutes ago   Up 28 minutes (unhealthy)   0.0.0.0:5555->5555/tcp, :::5555->5555/tcp, 8080/tcp   airflow_flower_1
a291cf238b9f   apache/airflow:2.0.2   "/usr/bin/dumb-init …"   31 minutes ago   Up 29 minutes               8080/tcp                                              airflow_airflow-init_1
fdb20e9152f3   apache/airflow:2.0.2   "/usr/bin/dumb-init …"   31 minutes ago   Up 29 minutes (unhealthy)   0.0.0.0:8080->8080/tcp, :::8080->8080/tcp             airflow_airflow-webserver_1
abf5a16aa846   apache/airflow:2.0.2   "/usr/bin/dumb-init …"   31 minutes ago   Up 29 minutes               8080/tcp                                              airflow_airflow-worker_1
f6dc352f407b   apache/airflow:2.0.2   "/usr/bin/dumb-init …"   31 minutes ago   Up 28 minutes               8080/tcp                                              airflow_airflow-scheduler_1
12dfc71e518f   redis:latest           "docker-entrypoint.s…"   31 minutes ago   Up 29 minutes (healthy)     0.0.0.0:6379->6379/tcp, :::6379->6379/tcp             airflow_redis_1

However the logs of one of them for example do not seem very informative.

# docker logs -f fdb20e9152f3
WARNING! You should run the image with GID (Group ID) set to 0
         even if you use 'airflow' user (UID=50000)
 You started the image with UID=50000 and GID=50000
 This is to make sure you can run the image with an arbitrary UID in the future.
 See more about it in the Airflow's docker image documentation
     http://airflow.apache.org/docs/docker-stack/entrypoint
BACKEND=postgresql+psycopg2
DB_HOST=my-db-endpoint
DB_PORT=5432
WARNING! You should run the image with GID (Group ID) set to 0
         even if you use 'airflow' user (UID=50000)
 You started the image with UID=50000 and GID=50000
 This is to make sure you can run the image with an arbitrary UID in the future.
 See more about it in the Airflow's docker image documentation
     http://airflow.apache.org/docs/docker-stack/entrypoint
BACKEND=postgresql+psycopg2
DB_HOST=my-db-endpoint
DB_PORT=5432

Regardless of any airflow - specific issues, how can I check docker - wise what's going on?

Docker seems to be aware of a couple of containers not being healty.

edit : both failing containers have the healtcheck condition

healthcheck:
      test: ["CMD", "curl", "--fail", "http://localhost:5555/"]

and

healthcheck:
      test: ["CMD", "curl", "--fail", "http://localhost:8080/"]

that seems to be failing by looking into their inspect output

 Failed to connect to localhost port 8080: Connection refused

but I cannot pinpoint what is causing the failure.

edit : I have tried following the instructions to start the init service first as well

# docker-compose up airflow-init
Starting airflow_redis_1 ... done
Starting airflow_airflow-init_1 ... done
Attaching to airflow_airflow-init_1
airflow-init_1       | BACKEND=postgresql+psycopg2
airflow-init_1       | DB_HOST=my-db-endpoint
airflow-init_1       | DB_PORT=5432

but it never exits, it prints the above message and that's it...

Answer 1

I ran into similar issue and it was docker volume causing the issue. As I was running lots of containers on my mac, there wasn;t enough disk space. I managed to fixed this issue my pruning the docker volume.

docker volume prune

This will remove any unused volume on your mac book. Before running this command please check if you got any useful data.

Answer 2

For docker-compose, from the entrypoint, the default value of group id is 0.

"${AIRFLOW_UID:-50000}:${AIRFLOW_GID:-0}"

Edit your docker-compose.yaml file or ad ass env.sh file in your Airflow project repository.

Answer 3

It does seems to be an error due to less memory allocated to docker for running this image, try to increase the resources available to docker and see the magic