Cannot access host machine from docker container

Question

Using docker version 18.09.2. Using docker on windows 10.

Setting up a prometheus and grafana stack to monitor metrics on a service running on my localhost. Here's my docker compose file.

version: '3.4'

networks:
  monitor-net:
    driver: bridge
  dockernet:
    external: true

volumes:
    prometheus_data: {}
    grafana_data: {}

services:

  prometheus:
    image: prom/prometheus:v2.7.1
    container_name: prometheus
    volumes:
      - ./prometheus/:/etc/prometheus/
      - prometheus_data:/prometheus
    command:
      - '--config.file=/etc/prometheus/prometheus.yml'
      - '--storage.tsdb.path=/prometheus'
      - '--web.console.libraries=/etc/prometheus/console_libraries'
      - '--web.console.templates=/etc/prometheus/consoles'
      - '--storage.tsdb.retention.time=200h'
      - '--web.enable-lifecycle'
    restart: unless-stopped
    expose:
      - 9090
    networks:
      - monitor-net
      - dockernet
    extra_hosts:
      - "localhost1:10.0.75.1"
    labels:
      org.label-schema.group: "monitoring"


  grafana:
    image: grafana/grafana:5.4.3
    container_name: grafana
    volumes:
      - grafana_data:/var/lib/grafana
      - ./grafana/datasources:/etc/grafana/datasources
      - ./grafana/dashboards:/etc/grafana/dashboards
      - ./grafana/setup.sh:/setup.sh
    entrypoint: /setup.sh
    environment:
      - GF_SECURITY_ADMIN_USER=${ADMIN_USER:-admin}
      - GF_SECURITY_ADMIN_PASSWORD=${ADMIN_PASSWORD:-admin}
      - GF_USERS_ALLOW_SIGN_UP=false
    restart: unless-stopped
    expose:
      - 3000
    networks:
      - monitor-net
      - dockernet
    labels:
      org.label-schema.group: "monitoring"

  caddy:
    image: stefanprodan/caddy
    container_name: caddy
    ports:
      - "3000:3000"
      - "9090:9090"
      - "9093:9093"
      - "9091:9091"
    volumes:
      - ./caddy/:/etc/caddy/
    environment:
      - ADMIN_USER=${ADMIN_USER:-admin}
      - ADMIN_PASSWORD=${ADMIN_PASSWORD:-admin}
    restart: unless-stopped
    networks:
      - monitor-net
      - dockernet
    labels:
      org.label-schema.group: "monitoring"

Here is my prometheus.yml file.

global:
  scrape_interval:     15s
  evaluation_interval: 15s

  # Attach these labels to any time series or alerts when communicating with
  # external systems (federation, remote storage, Alertmanager).
  external_labels:
      monitor: 'docker-host-alpha'

# Load and evaluate rules in this file every 'evaluation_interval' seconds.
rule_files:
  - "alert.rules"

# A scrape configuration containing exactly one endpoint to scrape.
scrape_configs:
  - job_name: 'prometheus'
    scrape_interval: 10s
    static_configs:
      - targets: ['localhost:9090']

  - job_name: 'myapp'
    scrape_interval: 10s 
    metrics_path: /metrics
    static_configs:
      - targets: ['docker.for.win.localhost:32771']

  - job_name: 'myapp1'
    scrape_interval: 10s 
    metrics_path: /metrics
    static_configs:
      - targets: ['docker.for.win.host.internal:51626']

  - job_name: 'myapp2'
    scrape_interval: 10s 
    metrics_path: /metrics
    static_configs:
      - targets: ['docker.for.win.host.internal.localhost:51626']

  - job_name: 'myapp3'
    scrape_interval: 10s 
    metrics_path: /metrics
    static_configs:
      - targets: ['docker.for.win.host.localhost:51626']

  - job_name: 'myapp4'
    scrape_interval: 10s 
    metrics_path: /metrics
    static_configs:
      - targets: ['docker.for.win.localhost:51626']

  - job_name: 'myapp5'
    scrape_interval: 10s 
    metrics_path: /metrics
    static_configs:
      - targets: ['host.docker.internal:51626']

  - job_name: 'myapp6'
    scrape_interval: 10s 
    metrics_path: /metrics
    static_configs:
      - targets: ['host.docker.internal.localhost:51626']

  - job_name: 'myapp7'
    scrape_interval: 10s 
    metrics_path: /metrics
    static_configs:
      - targets: ['docker.for.win.localhost:51626']

  - job_name: 'myapp8'
    scrape_interval: 10s 
    metrics_path: /metrics
    static_configs:
      - targets: ['127.0.0.1:51626']

  - job_name: 'myapp9'
    scrape_interval: 10s 
    metrics_path: /metrics
    static_configs:
      - targets: ['localhost:51626']

  - job_name: 'myapp10'
    scrape_interval: 10s 
    metrics_path: /metrics
    static_configs:
      - targets: ['10.0.75.1:51626']

  - job_name: 'myapp12'
    scrape_interval: 10s 
    metrics_path: /metrics
    static_configs:
      - targets: ['localhost1:51626']

From what I understand host.docker.internal should reference my host IP and give me access to my local app but it didn't. So then I looked up my docker NAT IP address with ipconfig (the 10.0.75.1 address) and that didn't work either.

Then I tried the network binding of localhost1 to 10.0.75.1. I tried setting up a bridge network called dockernet and connect that way and it didn't work. When I launch my app in a docker container I can get to it through "docker.for.win.localhost:32771" but this container can't access my remote database so that's why I need it to run local. Prometheus gives the following responses for some of the respective addresses:

Endpoint: Error
http://docker.for.win.localhost:32771/metrics:     UP
http://host.docker.internal:51626/metrics:     server returned HTTP status 400 Bad Request
http://docker.for.win.localhost:51626/metrics:     server returned HTTP status 400 Bad Request
http://host.docker.internal.localhost:51626/metrics:     Get http://host.docker.internal.localhost:51626/metrics: dial tcp: lookup host.docker.internal.localhost on 127.0.0.11:53: no such host
http://docker.for.win.host.internal.localhost:51626/metrics:     Get http://docker.for.win.host.internal.localhost:51626/metrics: dial tcp: lookup docker.for.win.host.internal.localhost on 127.0.0.11:53: no such host

I've tried everything and am out of ideas. Can anyone shed some light?

Answer 1

I have the similar problem. I locally run my own application on IIS Express on port 52562, and prometheus inside container show that http://docker.for.win.localhost:52562/metrics return 400 BAD Request. Problem was that IIS Express listen only for localhost, so I edit bindings in my applicationhost.config from

<binding protocol="http" bindingInformation="*:52562:localhost" />

to

<binding protocol="http" bindingInformation="*:52562:" />

and restart IIS Express.

This fixed the problem.

Answer 2

for my works this

version: "3"

networks:
  sandbox:
    driver: bridge

services:

  prometheus:
    restart: always
    image: prom/prometheus:v2.3.2
    volumes: ["./prometheus/prometheus.yml:/etc/prometheus/prometheus.yml"]
    ports: ["9090:9090"]
    extra_hosts: ["host.docker.internal:172.17.0.1"] # from Gateway bridge and added /etc/hosts
    networks: ["sandbox"]

  grafana:
     ....

Answer 3

You might also check if your windows 10 firewall is blocking the connection.

To disable the firewall completely:

netsh advfirewall set allprofiles state off

To allow a connection on a specific port:

New-NetFirewallRule -Protocol TCP -LocalPort 44369 -Direction Inbound -Action Allow -DisplayName "Allow network TCP on port 44369"