docker-compose not starting postgres and gives error

Question

I'm really new to Docker (also postgres) and still finding my feet. I get an error and can't seem to get one of my postgres services running, although when I start it, I'm able to access pgadmin and airflow via the browser. I think there is some sort of conflict happening but I'm not sure where. I have a docker-compose.yml file that starts a few containers, as well as the postgres one in question which has the servce name db :

version: '3.7'
services:
    postgres:
        image: postgres:9.6
        environment:
            - POSTGRES_USER=airflow
            - POSTGRES_PASSWORD=airflow
            - POSTGRES_DB=airflow
        logging:
            options:
                max-size: 10m
                max-file: "3"

    db:
        image: postgres:13.0-alpine
        restart: always
        environment:
            POSTGRES_DB: postgres
            POSTGRES_USER: admin_user
            POSTGRES_PASSWORD: secret_password
            # PGDATA: /var/lib/postgresql/data
        volumes:
            - ./db-data:/var/lib/postgresql/data
        ports:
            - "5433:5432"
         
    pgadmin:
        image: dpage/pgadmin4:4.27
        restart: always
        environment:
            PGADMIN_DEFAULT_EMAIL: admin_user@test_email.com
            PGADMIN_DEFAULT_PASSWORD: test_password
            PGADMIN_LISTEN_PORT: 1111
        ports:
        - "1111:1111"
        volumes:
        - pgadmin-data:/var/lib/pgadmin
        links:
        - "db:pgsql-server"

    webserver:
        image: l/custom_airflow:1.5
        container_name: l_custom_airflow
        restart: always
        depends_on:
            - postgres
        environment:
            - LOAD_EX=n
            - EXECUTOR=Local
        logging:
            options:
                max-size: 10m
                max-file: "3"
        volumes:
            - ./dags:/usr/local/airflow/dags
            - ./db-data:/usr/local/airflow/db-data
            - ./pgadmin-data:/usr/local/airflow/pgadmin-data
        ports:
            - "8080:8080"
        command: webserver
        healthcheck:
            test: ["CMD-SHELL", "[ -f /usr/local/airflow/airflow-webserver.pid ]"]
            interval: 30s
            timeout: 30s
            retries: 3

volumes:
    db-data: 
    pgadmin-data: 

The relevant part is this:

db:
    image: postgres:13.0-alpine
    restart: always
    environment:
        POSTGRES_DB: postgres
        POSTGRES_USER: admin_user
        POSTGRES_PASSWORD: secret_password
        # PGDATA: /var/lib/postgresql/data
    volumes:
        - ./db-data:/var/lib/postgresql/data
    ports:
        - "5433:5432"
     

[I already have two version of postgres on my local machine, and I saw that they use ports 5432 and then 5433, so it looks like the latest one goes to 5433. Similarly, I have another service (airflow) that depends on an older version of postgres to run, so I assume since that one comes first it takes 5432, and then the new postgres service I want will likely be mapped to 5433 as default - please correct me if I'm wrong]

But when I run docker-compose up -d and check my containers with docker container ls -a I see that this particular container is continuously restarting. I ran docker logs --tail 50 --follow --timestamps pipeline_5_db_1 (the container name for the db service) and I see the following error:

2020-10-28T08:46:29.730973000Z chmod: /var/lib/postgresql/data: Operation not permitted                                                                                                                                    2020-10-28T08:46:30.468640800Z chmod: /var/lib/postgresql/data: Operation not permitted                                                                                                                                    2020-10-28T08:46:31.048144200Z chmod: /var/lib/postgresql/data: Operation not permitted                                                                                                                                    2020-10-28T08:46:31.803571400Z chmod: /var/lib/postgresql/data: Operation not permitted                                                                                                                                    2020-10-28T08:46:32.957604600Z chmod: /var/lib/postgresql/data: Operation not permitted                                                                                                                                    2020-10-28T08:46:34.885928500Z chmod: /var/lib/postgresql/data: Operation not permitted                                                                                                                                    2020-10-28T08:46:38.479922200Z chmod: /var/lib/postgresql/data: Operation not permitted                                                                                                                                    2020-10-28T08:46:45.384436400Z chmod: /var/lib/postgresql/data: Operation not permitted                                                                                                                                    2020-10-28T08:46:58.612202300Z chmod: /var/lib/postgresql/data: Operation not permitted 

I googled the error and saw a couple of other SO posts but I can't see a clear explanation. This post and this post are a bit unclear to me (might be because I'm not so familiar), so I'm not sure how to use the responses to solve this issue.

Answer 1

You've got dbdata defined as a named volume at the bottom of the compose file but you're using ./dbdata within each service which is a bind mount. You might try using the named volume instead of the shared directory in your db and webserver services, like this:

    volumes:
        - db-data:/var/lib/postgresql/data

A bind mount should also work but can be troublesome if permissions on the mounted directory aren't quite right, which might be your problem.

The above also applies to pgadmin-data where the pgadmin service is using a named volume but webserver is using the bind mount (local directory). In fact, it's not clear why the webserver would need access to those data directories. Typically, a webserver would connect to the database via port 5432 (which doesn't even need to be mapped on the host). See for instance the bitnami/airflow docs on Docker Hub .