Running a nomad job for a docker container that Traefik can find

Question

I'm currently running a docker container with Traefik as the load balancer using the following docker-compose file:

services:
  loris:
    image: bdlss/loris-grok-docker
    labels: 
      - traefik.http.routers.loris.rule=Host(`loris.my_domain`)
      - traefik.http.routers.loris.tls=true
      - traefik.http.routers.loris.tls.certresolver=lets-encrypt
      - traefik.port=80
    networks: 
      - web

It is working fairly well. As part of one my first attempts using Nomad, I simply want to be able to start this container using a nomad job loris.nomad instead of using the docker-compose file.

The Docker container 'Labels' and the 'Network' identification are quite important for Traefik to do the dynamic routing.

My question is : where can I put this "label" information and "network" information in the loris.nomad file so that it starts the container in the same way that the docker-compose file currently does.

I've tried putting this information in the task.config stanza but this doesn't work and I'm having trouble following the documentation. I've seen examples where an additional "service" stanza has been added, but I"m still not sure.

Here's the basics of that nomad file I want to modify.

# loris.nomad

job "loris" {
  datacenters = ["dc1"]
  group "loris" {
    network {
      port "http" {
        to = 5004
    }
  
    task "loris" {
      driver = "docker"

      config {
        image = "bdlss/loris-openjpeg-docker"
        ports = ["http"]
      }

      resources {
        cpu    = 500
        memory = 512
      }
    }
  }
}

Any advice is much appreciated.

Answer 1

Well, the most appropriate option for running traefik in nomad and load-balance between containers is using consul catalog (required for service discovery).

For this to run you have to confgure the consule connection when you start nomad. If you like to test things out locally you can do this by simply running sudo nomad agent -dev-connect . Consul can be started with consul agent -dev -client="0.0.0.0" .

Now you can simply provide your traefik configuration using tags as it is shown here .

If you really need (which will cause issues in a clustered setup for sure) to run traefik in nomad with docker provider you can do the following:

First you need to enable host path mounting in the docker plugin. See this and this . You can place your configuration in an extra file like extra.hcl which looks like this:

plugin "docker" {
  config {
    volumes {
      enabled = true
    }
  }
}

Now you can start nomad with this extra setting sudo nomad agent -dev-connect -config=extra.hcl . Now you can provide your traefik settings in the config/labels block, like (full):

job "traefik" {
  region      = "global"
  datacenters = ["dc1"]
  type        = "service"

  group "traefik" {
    count = 1

    task "traefik" {
      driver = "docker"

      config {
        image        = "traefik:v2.3"
        //network_mode = "host"

        volumes = [
          "local/traefik.yaml:/etc/traefik/traefik.yaml",
          "/var/run/docker.sock:/var/run/docker.sock"
        ]

        labels {
          traefik.enable = true
          traefik.http.routers.from-docker.rule = "Host(`docker.loris.mydomain`)"
          traefik.http.routers.from-docker.entrypoints = "web"
          traefik.http.routers.from-docker.service = "api@internal"
        }
      }

      template {
        data = <<EOF
log:
  level: DEBUG

entryPoints:
  traefik:
    address: ":8080"
  web:
    address: ":80"

api:
  dashboard: true
  insecure: true

accessLog: {}

providers:
  docker:
    exposedByDefault: false  
  consulCatalog:
    prefix: "traefik"
    exposedByDefault: false
    endpoint:
      address: "10.0.0.20:8500"
      scheme: "http"
      datacenter: "dc1"
EOF

        destination = "local/traefik.yaml"
      }

      resources {
        cpu    = 100
        memory = 128

        network {
          mbits = 10

          port "http" {
            static = 80
          }

          port "traefik" {
            static = 8080
          }
        }
      }

      service {
        name = "traefik"
        tags = [
          "traefik.enable=true",
          "traefik.http.routers.from-consul.rule=Host(`consul.loris.mydomain`)",
          "traefik.http.routers.from-consul.entrypoints=web",
          "traefik.http.routers.from-consul.service=api@internal"
        ]

        check {
          name     = "alive"
          type     = "tcp"
          port     = "http"
          interval = "10s"
          timeout  = "2s"
        }
      }
    }
  }
}

(There might be a setting to bind to 0.0.0.0 I defined those domains in my /etc/hosts to point to my main interface IP).

You can test it with this modified webapp spec (I didn't figure out how to map ports correctly, like container:80 -> host:<random> , but I think it is enough to show how complicated it gets:)):

job "demo-webapp" {
  datacenters = ["dc1"]

  group "demo" {
    count = 3

    task "server" {
      env {
        // "${NOMAD_PORT_http}"
        PORT    = "80"
        NODE_IP = "${NOMAD_IP_http}"
      }

      driver = "docker"

      config {
        image = "hashicorp/demo-webapp-lb-guide"

        labels {
          traefik.enable = true
          traefik.http.routers.webapp-docker.rule = "Host(`docker.loris.mydomain`) && Path(`/myapp`)"
          traefik.http.services.webapp-docker.loadbalancer.server.port = 80
        }
      }

      resources {
        network {
          // Used for docker provider
          mode ="bridge"
          mbits = 10
          port  "http"{
            // Used for docker provider
            to = 80
          }
        }
      }

      service {
        name = "demo-webapp"
        port = "http"

        tags = [
          "traefik.enable=true",
          "traefik.http.routers.webapp-consul.rule=Host(`consul.loris.mydomain`) && Path(`/myapp`)",
        ]

        check {
          type     = "http"
          path     = "/"
          interval = "2s"
          timeout  = "2s"
        }
      }
    }
  }
}

I hope this somehow answers your question.