Question

In my docker-compose.yml (version - 2.3), I have setup a healthcheck for couple of services. Here is the relevant yml snippet -

      healthcheck:
        test: ["CMD-SHELL", "/secrets/authz_Startup.sh"]
        interval: 30s
        timeout: 30s
        retries: 3
        start_period: 300s 

Here is my authz_Startup.sh content -

#!/bin/bash
HTTP_STATUS="$(curl -IL --silent https://myHost.com/authz/ping 2>/dev/null | head -n 1 | cut -d ' ' -f 2)";
if [ "$HTTP_STATUS" = "200" ]; then
    echo "Service started";
    exit 0;
else
    echo "***Service NOT started*** --> $HTTP_STATUS";  
    exit 1;
fi
exit;

However, when I run "docker-compose up" and then inspect the health check output using "docker inspect --format='{{json .State.Health}}' CONTAINER_ID | jq", I always get -

{
  "Status": "unhealthy",
  "FailingStreak": 16,
  "Log": [
    {
      "Start": "2018-03-10T16:05:18.361126144+05:30",
      "End": "2018-03-10T16:05:18.668852098+05:30",
      "ExitCode": 1,
      "Output": "***Service NOT started*** --> \n"
    },
    {
      "Start": "2018-03-10T16:05:48.676933885+05:30",
      "End": "2018-03-10T16:05:48.973643139+05:30",
      "ExitCode": 1,
      "Output": "***Service NOT started*** --> \n"
    }
  ]
}

I am however able to run this script from terminal and get "Service started" message. I am also able to see a "200 OK" response when I request the ping url from my browser and the typical response time is in ms. Also, I am able to see the 200 response on my browser much before the service is declared "unhealthy".

I am at my wits end trying to figure out whatever it is that I am not doing right. Can someone please help me with this?

Answer 1

Figured this out --> In my setup, I have 4 docker containers running tomcat servers on port 8080 and mapped to ports (9000, 9001, 9002 & 9003) and an Apache Webserver running on port 443 on my ubuntu (v16.04) server. Apache has a reverse proxy configured to forward requests to individual docker containers. This part works fine. Eg - I make a request to apache webserver ( https://myHost.com/authz/ping ) and the request forwards to the individual docker container ( http://myHost.com:9000/authz/ping ).

However, healthcheck apis run from within docker containers and all this while I was attempting to make this request --> https://myHost.com/authz/ping but the docker container is unable to connect to the Apache webserver running on the host machine and that is why it was failing.

I changed the healthcheck url to http://localhost:9000/authz/ping and it started working fine. So, this bit is now sorted but I still need to figure out some way to make services from within docker container, discover services running on host machines.