Traefik requests sub-domain specific certificate instead of wildcard?

Question

I have the following docker-compose.yml file

version: "3.8"

secrets:
  loopia_api_user:
    file: "./traefik/secrets/loopia_api_user.secret"
  loopia_api_password:
    file: "./traefik/secrets/loopia_api_password.secret"

networks:
  dockersocket:
    driver: bridge
    internal: true
  traefik:
    external: true  # this network has to be created once before starting:
    name: traefik   # docker network create traefik

services:
  docker-socket-proxy:
    image: tecnativa/docker-socket-proxy
    container_name: docker-socket-proxy
    restart: unless-stopped
    mem_limit: 128M
    cpus: 0.5
    networks:
      - dockersocket
    expose:
      - 2375
    environment:
      CONTAINERS: 1
    volumes:
      - /var/run/docker.sock:/var/run/docker.sock:ro

  traefik:
    image: "traefik:${TRAEFIK_VERSION}"
    container_name: "traefik"
    restart: unless-stopped
    read_only: true
    mem_limit: 2G
    cpus: 0.75
    security_opt:
      - no-new-privileges:true
    depends_on:
      - docker-socket-proxy
    secrets:
      - "loopia_api_user"
      - "loopia_api_password"
    command:
      - "--log.level=DEBUG"
      - "--api.insecure=true"
      - "--providers.docker=true"
      - "--providers.docker.exposedbydefault=false"
      - "--entrypoints.web.address=:80"
      - "--entrypoints.websecure.address=:443"
      #- "--entrypoints.web.http.redirections.entryPoint.to=websecure" # Redirect http to https
      #- "--entrypoints.web.http.redirections.entrypoint.scheme=https"
      # Https configuration
      - "--entrypoints.websecure.http.tls=true"
      - "--entrypoints.websecure.http.tls.domains[0].main=${DOMAIN}"
      - "--entrypoints.websecure.http.tls.domains[0].sans=*.${DOMAIN}"
      - "--entrypoints.websecure.http.tls.certresolver=loopia"
      # Lets encrypt Loopia dns challange
      - "--certificatesresolvers.loopia.acme.caserver=https://acme-staging-v02.api.letsencrypt.org/directory"
      - "--certificatesresolvers.loopia.acme.dnschallenge.resolvers=1.1.1.1:53,8.8.8.8:53"
      - "--certificatesresolvers.loopia.acme.dnschallenge=true"
      - "--certificatesresolvers.loopia.acme.dnschallenge.provider=loopia"
      - "--certificatesresolvers.loopia.acme.email=${POSTMASTER_EMAIL}"
      - "--certificatesresolvers.loopia.acme.storage=/letsencrypt/acme.json"
      # Use the docker socket proxy
      - "--providers.docker.endpoint=tcp://docker-socket-proxy:2375" #  using Docker Socket Proxy instead of docker socket for improved security
      - "--providers.docker.network=traefik" # Defines a default docker network to use for connections to all containers.
      # Logs
      - --accesslog.filepath=/logs/access.log
      - --accesslog.format=json
      - --accesslog.fields.defaultMode=keep
      - --accesslog.fields.headers.defaultMode=keep
      - --log.filepath=/logs/traefik.log
    environment:
      - LOOPIA_API_USER_FILE=/run/secrets/loopia_api_user
      - LOOPIA_API_PASSWORD_FILE=/run/secrets/loopia_api_password
      - DOMAIN
    ports:
      - "80:80"
      - "443:443"
      - "8080:8080"
    volumes:
      - ./traefik/logs:/logs
      - ./traefik/dynamic_conf:/dynamic_conf:ro
      - ./traefik/letsencrypt:/letsencrypt
    networks:
      - dockersocket
      - traefik

  whoami:
    image: "traefik/whoami"
    container_name: "simple-service"
    labels:
      - "traefik.enable=true"
      - "traefik.http.routers.whoami.rule=Host(`whoami.${DOMAIN}`)"
      - "traefik.http.routers.whoami.tls.certresolver=loopia"
      - "traefik.http.routers.whoami.entrypoints=websecure"
    networks:
      - traefik

TRAEFIK_VERSION is v2.9.6 .

When I start my containers traefik requests a certificate as expected but does so for whoami.${DOMAIN} instead of *.${DOMAIN} what am I missing?

I filtered the log somewhat so if any line is missing tell me and ill fetch it.

time="2023-01-29T21:12:23+01:00" level=debug msg="Static configuration loaded {\"global\":{\"checkNewVersion\":true},\"serversTransport\":{\"maxIdleConnsPerHost\":200},\"entryPoints\":{\"traefik\":{\"address\":\":8080\",\"transport\":{\"lifeCycle\":{\"graceTimeOut\":\"10s\"},\"respondingTimeouts\":{\"idleTimeout\":\"3m0s\"}},\"forwardedHeaders\":{},\"http\":{},\"http2\":{\"maxConcurrentStreams\":250},\"udp\":{\"timeout\":\"3s\"}},\"web\":{\"address\":\":80\",\"transport\":{\"lifeCycle\":{\"graceTimeOut\":\"10s\"},\"respondingTimeouts\":{\"idleTimeout\":\"3m0s\"}},\"forwardedHeaders\":{},\"http\":{\"redirections\":{\"entryPoint\":{\"to\":\"websecure\",\"scheme\":\"https\",\"permanent\":true,\"priority\":2147483646}}},\"http2\":{\"maxConcurrentStreams\":250},\"udp\":{\"timeout\":\"3s\"}},\"websecure\":{\"address\":\":443\",\"transport\":{\"lifeCycle\":{\"graceTimeOut\":\"10s\"},\"respondingTimeouts\":{\"idleTimeout\":\"3m0s\"}},\"forwardedHeaders\":{},\"http\":{\"tls\":{\"certResolver\":\"loopia\",\"domains\":[{\"main\":\"example.com\",\"sans\":[\"*.example.com\"]}]}},\"http2\":{\"maxConcurrentStreams\":250},\"udp\":{\"timeout\":\"3s\"}}},\"providers\":{\"providersThrottleDuration\":\"2s\",\"docker\":{\"watch\":true,\"endpoint\":\"tcp://docker-socket-proxy:2375\",\"defaultRule\":\"Host(`{{ normalize .Name }}`)\",\"network\":\"traefik\",\"swarmModeRefreshSeconds\":\"15s\"},\"file\":{\"directory\":\"/dynamic_conf/\",\"watch\":true}},\"api\":{\"insecure\":true,\"dashboard\":true},\"log\":{\"level\":\"DEBUG\",\"filePath\":\"/logs/traefik.log\",\"format\":\"common\"},\"accessLog\":{\"filePath\":\"/logs/access.log\",\"format\":\"json\",\"filters\":{},\"fields\":{\"defaultMode\":\"keep\",\"headers\":{\"defaultMode\":\"keep\"}}},\"certificatesResolvers\":{\"loopia\":{\"acme\":{\"email\":\"peter@example.com\",\"caServer\":\"https://acme-staging-v02.api.letsencrypt.org/directory\",\"storage\":\"/letsencrypt/acme.json\",\"keyType\":\"RSA4096\",\"certificatesDuration\":2160,\"dnsChallenge\":{\"provider\":\"loopia\",\"resolvers\":[\"1.1.1.1:53\",\"8.8.8.8:53\"]}}}}}"
time="2023-01-29T21:12:23+01:00" level=info msg="Starting provider *acme.ChallengeTLSALPN"
time="2023-01-29T21:12:23+01:00" level=debug msg="*acme.ChallengeTLSALPN provider configuration: {}"
time="2023-01-29T21:12:23+01:00" level=info msg="Starting provider *acme.Provider"
time="2023-01-29T21:12:23+01:00" level=debug msg="*acme.Provider provider configuration: {\"email\":\"peter@example.com\",\"caServer\":\"https://acme-staging-v02.api.letsencrypt.org/directory\",\"storage\":\"/letsencrypt/acme.json\",\"keyType\":\"RSA4096\",\"certificatesDuration\":2160,\"dnsChallenge\":{\"provider\":\"loopia\",\"resolvers\":[\"1.1.1.1:53\",\"8.8.8.8:53\"]},\"ResolverName\":\"loopia\",\"store\":{},\"TLSChallengeProvider\":{},\"HTTPChallengeProvider\":{}}"
time="2023-01-29T21:12:23+01:00" level=debug msg="Configuration received: {\"http\":{\"middlewares\":{\"https-redirect\":{\"redirectScheme\":{\"scheme\":\"https\"}},\"local-only\":{\"ipWhiteList\":{\"sourceRange\":[\"127.0.0.1/32\",\"192.168.0.0/24\",\"172.20.0.0/24\"]}},\"securedheaders\":{\"headers\":{\"customResponseHeaders\":{\"X-Robots-Tag\":\"none,noarchive,nosnippet,notranslate,noimageindex,\",\"server\":\"\"},\"sslRedirect\":true,\"stsSeconds\":63072000,\"stsIncludeSubdomains\":true,\"stsPreload\":true,\"forceSTSHeader\":true,\"customFrameOptionsValue\":\"SAMEORIGIN\",\"contentTypeNosniff\":true,\"browserXssFilter\":true,\"referrerPolicy\":\"same-origin\",\"featurePolicy\":\"camera 'none'; geolocation 'none'; microphone 'none'; payment 'none'; usb 'none'; vr 'none';\"}}}},\"tcp\":{},\"udp\":{},\"tls\":{\"options\":{\"default\":{\"minVersion\":\"VersionTLS13\",\"clientAuth\":{},\"sniStrict\":true,\"alpnProtocols\":[\"h2\",\"http/1.1\",\"acme-tls/1\"]},\"mintls13\":{\"minVersion\":\"VersionTLS13\",\"clientAuth\":{},\"sniStrict\":true,\"alpnProtocols\":[\"h2\",\"http/1.1\",\"acme-tls/1\"]}}}}" providerName=file
time="2023-01-29T21:12:23+01:00" level=debug msg="Attempt to renew certificates \"720h0m0s\" before expiry and check every \"24h0m0s\"" ACME CA="https://acme-staging-v02.api.letsencrypt.org/directory" providerName=loopia.acme
time="2023-01-29T21:12:23+01:00" level=info msg="Testing certificate renew..." ACME CA="https://acme-staging-v02.api.letsencrypt.org/directory" providerName=loopia.acme
time="2023-01-29T21:12:23+01:00" level=debug msg="Configuration received: {\"http\":{\"routers\":{\"api\":{\"entryPoints\":[\"traefik\"],\"service\":\"api@internal\",\"rule\":\"PathPrefix(`/api`)\",\"priority\":2147483646},\"dashboard\":{\"entryPoints\":[\"traefik\"],\"middlewares\":[\"dashboard_redirect@internal\",\"dashboard_stripprefix@internal\"],\"service\":\"dashboard@internal\",\"rule\":\"PathPrefix(`/`)\",\"priority\":2147483645},\"web-to-websecure\":{\"entryPoints\":[\"web\"],\"middlewares\":[\"redirect-web-to-websecure\"],\"service\":\"noop@internal\",\"rule\":\"HostRegexp(`{host:.+}`)\",\"priority\":2147483646}},\"services\":{\"api\":{},\"dashboard\":{},\"noop\":{}},\"middlewares\":{\"dashboard_redirect\":{\"redirectRegex\":{\"regex\":\"^(http:\\\\/\\\\/(\\\\[[\\\\w:.]+\\\\]|[\\\\w\\\\._-]+)(:\\\\d+)?)\\\\/$\",\"replacement\":\"${1}/dashboard/\",\"permanent\":true}},\"dashboard_stripprefix\":{\"stripPrefix\":{\"prefixes\":[\"/dashboard/\",\"/dashboard\"]}},\"redirect-web-to-websecure\":{\"redirectScheme\":{\"scheme\":\"https\",\"port\":\"443\",\"permanent\":true}}},\"models\":{\"websecure\":{\"tls\":{\"certResolver\":\"loopia\",\"domains\":[{\"main\":\"example.com\",\"sans\":[\"*.example.com\"]}]}}},\"serversTransports\":{\"default\":{\"maxIdleConnsPerHost\":200}}},\"tcp\":{},\"udp\":{},\"tls\":{}}" providerName=internal
time="2023-01-29T21:12:23+01:00" level=debug msg="Configuration received: {\"http\":{},\"tcp\":{},\"udp\":{},\"tls\":{}}" providerName=loopia.acme
time="2023-01-29T21:12:23+01:00" level=debug msg="Configuration received: {\"http\":{\"routers\":{\"whoami\":{\"entryPoints\":[\"websecure\"],\"service\":\"whoami-docker\",\"rule\":\"Host(`whoami.example.com`)\",\"tls\":{\"certResolver\":\"loopia\"}}},\"services\":{\"whoami-docker\":{\"loadBalancer\":{\"servers\":[{\"url\":\"http://172.20.0.2:80\"}],\"passHostHeader\":true}}}},\"tcp\":{},\"udp\":{}}" providerName=docker
time="2023-01-29T21:12:23+01:00" level=debug msg="No default certificate, fallback to the internal generated certificate" tlsStoreName=default
time="2023-01-29T21:12:23+01:00" level=debug msg="No default certificate, fallback to the internal generated certificate" tlsStoreName=default
time="2023-01-29T21:12:23+01:00" level=debug msg="Adding route for whoami.example.com with TLS options default" entryPointName=websecure
time="2023-01-29T21:12:23+01:00" level=debug msg="Trying to challenge certificate for domain [whoami.example.com] found in HostSNI rule" providerName=loopia.acme routerName=whoami@docker rule="Host(`whoami.example.com`)" ACME CA="https://acme-staging-v02.api.letsencrypt.org/directory"
time="2023-01-29T21:12:23+01:00" level=debug msg="Looking for provided certificate(s) to validate [\"whoami.example.com\"]..." routerName=whoami@docker rule="Host(`whoami.example.com`)" ACME CA="https://acme-staging-v02.api.letsencrypt.org/directory" providerName=loopia.acme
time="2023-01-29T21:12:23+01:00" level=debug msg="Domains [\"whoami.example.com\"] need ACME certificates generation for domains \"whoami.example.com\"." routerName=whoami@docker rule="Host(`whoami.example.com`)" ACME CA="https://acme-staging-v02.api.letsencrypt.org/directory" providerName=loopia.acme
time="2023-01-29T21:12:23+01:00" level=debug msg="Loading ACME certificates [whoami.example.com]..." providerName=loopia.acme routerName=whoami@docker rule="Host(`whoami.example.com`)" ACME CA="https://acme-staging-v02.api.letsencrypt.org/directory"
time="2023-01-29T21:12:24+01:00" level=debug msg="Building ACME client..." providerName=loopia.acme
time="2023-01-29T21:12:24+01:00" level=debug msg="https://acme-staging-v02.api.letsencrypt.org/directory" providerName=loopia.acme
time="2023-01-29T21:12:24+01:00" level=info msg=Register... providerName=loopia.acme
time="2023-01-29T21:12:24+01:00" level=debug msg="legolog: [INFO] acme: Registering account for peter@example.com"
time="2023-01-29T21:12:25+01:00" level=debug msg="Using DNS Challenge provider: loopia" providerName=loopia.acme
time="2023-01-29T21:12:25+01:00" level=debug msg="legolog: [INFO] [whoami.example.com] acme: Obtaining bundled SAN certificate"
time="2023-01-29T21:12:25+01:00" level=debug msg="Configuration received: {\"http\":{\"routers\":{\"whoami\":{\"entryPoints\":[\"websecure\"],\"service\":\"whoami-docker\",\"rule\":\"Host(`whoami.example.com`)\",\"tls\":{\"certResolver\":\"loopia\"}}},\"services\":{\"whoami-docker\":{\"loadBalancer\":{\"servers\":[{\"url\":\"http://172.20.0.2:80\"}],\"passHostHeader\":true}}}},\"tcp\":{},\"udp\":{}}" providerName=docker
time="2023-01-29T21:12:25+01:00" level=debug msg="legolog: [INFO] [whoami.example.com] AuthURL: https://acme-staging-v02.api.letsencrypt.org/acme/authz-v3/5153870843"
time="2023-01-29T21:12:25+01:00" level=debug msg="legolog: [INFO] [whoami.example.com] acme: Could not find solver for: tls-alpn-01"
time="2023-01-29T21:12:25+01:00" level=debug msg="legolog: [INFO] [whoami.example.com] acme: Could not find solver for: http-01"
time="2023-01-29T21:12:25+01:00" level=debug msg="legolog: [INFO] [whoami.example.com] acme: use dns-01 solver"
time="2023-01-29T21:12:25+01:00" level=debug msg="legolog: [INFO] [whoami.example.com] acme: Preparing to solve DNS-01"
time="2023-01-29T21:12:26+01:00" level=debug msg="legolog: [INFO] [whoami.example.com] acme: Trying to solve DNS-01"
time="2023-01-29T21:12:26+01:00" level=debug msg="legolog: [INFO] [whoami.example.com] acme: Checking DNS record propagation using [1.1.1.1:53 8.8.8.8:53]"
time="2023-01-29T21:13:31+01:00" level=debug msg="legolog: [INFO] [whoami.example.com] acme: Cleaning DNS-01 challenge"
time="2023-01-29T21:13:32+01:00" level=debug msg="legolog: [INFO] [whoami.example.com] acme: Validations succeeded; requesting certificates"
time="2023-01-29T21:13:33+01:00" level=debug msg="Certificates obtained for domains [whoami.example.com]" providerName=loopia.acme routerName=whoami@docker rule="Host(`whoami.example.com`)" ACME CA="https://acme-staging-v02.api.letsencrypt.org/directory"
time="2023-01-29T21:13:33+01:00" level=debug msg="Configuration received: {\"http\":{},\"tcp\":{},\"udp\":{},\"tls\":{}}" providerName=loopia.acme
time="2023-01-29T21:13:33+01:00" level=debug msg="No default certificate, fallback to the internal generated certificate" tlsStoreName=default
time="2023-01-29T21:13:33+01:00" level=debug msg="Adding route for whoami.example.com with TLS options default" entryPointName=websecure
time="2023-01-29T21:13:33+01:00" level=debug msg="Trying to challenge certificate for domain [whoami.example.com] found in HostSNI rule" routerName=whoami@docker rule="Host(`whoami.example.com`)" ACME CA="https://acme-staging-v02.api.letsencrypt.org/directory" providerName=loopia.acme
time="2023-01-29T21:13:33+01:00" level=debug msg="Looking for provided certificate(s) to validate [\"whoami.example.com\"]..." ACME CA="https://acme-staging-v02.api.letsencrypt.org/directory" providerName=loopia.acme routerName=whoami@docker rule="Host(`whoami.example.com`)"
time="2023-01-29T21:13:33+01:00" level=debug msg="No ACME certificate generation required for domains [\"whoami.example.com\"]." ACME CA="https://acme-staging-v02.api.letsencrypt.org/directory" providerName=loopia.acme routerName=whoami@docker rule="Host(`whoami.example.com`)"

acme.json

{
  "loopia": {
    "Account": {
      "Email": "peter@example.com",
      "Registration": {
        "body": {
          "status": "valid",
          "contact": [
            "mailto:peter@example.com"
          ]
        },
        "uri": "https://acme-staging-v02.api.letsencrypt.org/acme/acct/85276103"
      },
      "PrivateKey": "....",
      "KeyType": "4096"
    },
    "Certificates": [
      {
        "domain": {
          "main": "whoami.example.com"
        },
        "certificate": "....",
        "key": "....",
        "Store": "default"
      }
    ]
  }
}

Edit: I even tried to add to add a second certificate but that just generated 2 certificates instead.

Answer 1

Adding the following to the traefik container solved the issue

labels:
  - "traefik.enable=true"
  - 'traefik.http.routers.wildcard-certs.tls.certresolver=loopia'
  - 'traefik.http.routers.wildcard-certs.tls.domains[0].main=${DOMAIN}'
  - 'traefik.http.routers.wildcard-certs.tls.domains[0].sans=*.${DOMAIN}'

But is

- "--entrypoints.websecure.http.tls.domains[0].main=${DOMAIN}"
- "--entrypoints.websecure.http.tls.domains[0].sans=*.${DOMAIN}"
- "--entrypoints.websecure.http.tls.certresolver=loopia"

Incorrect? (most blog entries online use those commands)