Bash redirection in docker container failing when ran in ECS task on Amazon Linux 2 instances

Question

I am trying to run an ECS task that contains 3 containers - postgres, redis, and an image from a private ECR repository. The custom image container definition has a command to wait until the postgres container can receive traffic via a bash command

"command": [
    "/bin/bash",
    "-c",
    "while !</dev/tcp/postgres/5432; do echo \"Waiting for postgres database to start...\"; /bin/sleep 1; done; /bin/sh /app/start-server.sh;"
],

When I run this via docker-compose on my local machine through docker it works, but on the Amazon Linux 2 EC2 machine this is printed when the while loop runs:

/bin/bash: line 1: postgres: Name or service not known

/bin/bash: line 1: /dev/tcp/postgres/5432: Invalid argument

The postgres container runs without error and the last log from that container is

database system is ready to accept connections

I am not sure if this is a docker network issue or an issue with amazon linux 2's bash not being compiled with --enable-net-redirections which I found explained here

Task Definition:

{
    "networkMode": "bridge",
    "containerDefinitions": [
        {
            "environment": [
                {
                    "name": "POSTGRES_DB",
                    "value": "metadeploy"
                },
                {
                    "name": "POSTGRES_USER",
                    "value": "<redacted>"
                },
                {
                    "name": "POSTGRES_PASSWORD",
                    "value": "<redacted>"
                }
            ],
            "essential": true,
            "image": "postgres:12.9",
            "mountPoints": [],
            "name": "postgres",
            "memory": 1024,
            "logConfiguration": {
                "logDriver": "awslogs",
                "options": {
                    "awslogs-group": "metadeploy-postgres",
                    "awslogs-region": "us-east-1",
                    "awslogs-create-group": "true",
                    "awslogs-stream-prefix": "mdp"
                }
            }
        },
        {
            "essential": true,
            "image": "redis:6.2",
            "name": "redis",
            "memory": 1024
        },
        {
        "command": [
            "/bin/bash",
            "-c",
            "while !</dev/tcp/postgres/5432; do echo \"Waiting for postgres database to start...\"; /bin/sleep 1; done; /bin/sh /app/start-server.sh;"
        ],
            "environment": [
                {
                    "name": "DJANGO_SETTINGS_MODULE",
                    "value": "config.settings.local"
                },
                {
                    "name": "DATABASE_URL",
                    "value": "<redacted-postgres-url>"
                },
                {
                    "name": "REDIS_URL",
                    "value": "redis://redis:6379"
                },
                {
                    "name": "REDIS_HOST",
                    "value": "redis"
                }
            ],
            "essential": true,
            "image": "the private ecr image uri built from here https://github.com/SFDO-Tooling/MetaDeploy",
            "links": [
                "redis"
            ],
            "mountPoints": [
                {
                    "containerPath": "/app/node_modules",
                    "sourceVolume": "AppNode_Modules"
                }
            ],
            "name": "web",
            "portMappings": [
                {
                    "containerPort": 8080,
                    "hostPort": 8080
                },
                {
                    "containerPort": 8000,
                    "hostPort": 8000
                },
                {
                    "containerPort": 6006,
                    "hostPort": 6006
                }
            ],
            "memory": 1024,
            "logConfiguration": {
                "logDriver": "awslogs",
                "options": {
                    "awslogs-group": "metadeploy-web",
                    "awslogs-region": "us-east-1",
                    "awslogs-create-group": "true",
                    "awslogs-stream-prefix": "mdw"
                }
            }
        }
    ],
    "family": "MetaDeploy",
    "volumes": [
        {
            "host": {
                "sourcePath": "/app/node_modules"
            },
            "name": "AppNode_Modules"
        }
    ]
}

The corresponding docker-compose.yml contains:

version: '3'

services:
  postgres:
    environment:
      POSTGRES_DB: metadeploy
      POSTGRES_USER: postgres
      POSTGRES_PASSWORD: sample_db_password
    volumes:
      - ./postgres:/var/lib/postgresql/data:delegated
    image: postgres:12.9
    restart: always

  redis:
    image: redis:6.2

  web:
    build:
      context: .
      dockerfile: Dockerfile
    command: |
      /bin/bash -c 'while !</dev/tcp/postgres/5432; do echo "Waiting for postgres database to start..."; /bin/sleep 1; done; \
      /bin/sh /app/start-server.sh;'
    ports:
      - '8080:8080'
      - '8000:8000'
      # Storybook server
      - '6006:6006'
    stdin_open: true
    tty: true
    depends_on:
      - postgres
      - redis
    links:
      - redis
    environment:
      DJANGO_SETTINGS_MODULE: config.settings.local
      DATABASE_URL: postgres://postgres:sample_db_password@postgres:5432/metadeploy
      REDIS_URL: redis://redis:6379
      REDIS_HOST: redis
    volumes:
      - .:/app:cached
      - /app/node_modules

Do I need to recompile bash to use --enable-net-redirections , and if so how can I do that?

Answer 1

Without bash's net redirection feature, your best bet is to use something like nc or netcat (if available) to determine if the port is open. If those aren't available, it may be worth modifying your app logic to better handle database failure cases.

Alternately, a potential better approach would be:

1. Adding ahealthcheck to the postgres image.
2. Modifying the web service's depends_on clause "long syntax" to add a dependency on postgres being service_healthy instead of the default service_started .

This approach has two key benefits:

1. The postgres image likely has the tools to detect if the database is up and running.
2. The web service no longer needs to manually check if the database is ready or not.