How do I run a docker compose file with modified telegraf config file?

Question

I'm trying to run a docker compose file on MacOS to run Telegraf, Mosquitto (MQTT), Grafana and InfluxDB. I'm trying to run Telegraf with a modified config file. The ultimate aim is to store and display data being sent from an arduino muscle sensor.

The docker compose file currently looks like this:

version: '3'

services: 
  influxdb:
    container_name: influxdb
    image: influxdb
    ports:
        - "8086:8086"
    environment:
      - INFLUXDB_DB=sensors
      - INFLUXDB_ADMIN_USER=telegraf
      - INFLUXDB_ADMIN_PASSWORD=telegraf
    restart: always

  telegraf: 
    image: telegraf
    restart: always
    ports: 
      - "5050:5050"
    volumes:
    - $PWD/telegraf.conf:/etc/telegraf/telegraf.conf:ro  

  grafana: 
    container_name: grafana
    image: grafana/grafana
    links:
      - influxdb
    hostname: grafana
    ports:
        - "3000:3000"

  mosquitto:
    container_name: mosquitto
    image: eclipse-mosquitto
    ports:
      - "1883:1883"
      - "9001:9001"
    depends_on: 
      - influxdb
    restart: always

I can run the build command and Mosquitto, Grafana and InfluxDB all seem to run however there are a number of issues with Telegraf. Regardless of what changes are made to volume in the compose file the default config file for Telegraf is used, as opposed to my modified config, which seems to prevent data being sent to InfluxDB.

The Telegraf post to InfluxDB error looks as follows:

telegraf     | 2020-03-03T11:40:49Z E! [outputs.influxdb] When writing to [http://localhost:8086]: Post http://localhost:8086/write?db=telegraf: dial tcp 127.0.0.1:8086: connect: connection refused
telegraf     | 2020-03-03T11:40:49Z E! [agent] Error writing to outputs.influxdb: could not write any address

Mosquitto seems to work as the MQTT.fx app is able to connect and publish/subscribe to the container. However there are regular connections made and dropped with an unknown name.

The following connection error continuously logs:

mosquitto    | 1583247033: New connection from 172.25.0.1 on port 1883.
mosquitto    | 1583247033: Client <unknown> disconnected due to protocol error.

I've considered writing a Telegraf dockerfile to set the config file however this seems like overkill as my understanding is that the volume section of the compose file should allow this config file to be used.

My telegraf.conf file is in the same directory as the docker-compose.yml file.

The question is a) Is my belief that the container is using the default telegraf file correct b) How do I get the altered telegraf.conf file to on the container

Answer 1

Without having your telegraf config file, it's not easy to tell if it's being loaded correctly and there's a network issue...

I find instead of using: $PWD/telegraf.conf:/etc/telegraf/telegraf.conf:ro , it makes more sense to include config files for docker in a local subdirectory: .docker/telegraf/telegraf.conf:/etc/telegraf/telegraf.conf:ro .

If you want to use a full path with PWD, I suggest ${PWD} . Based on my experiences, $PWD is using a variable from your terminal (you can test if it's assigned with echo $PWD ), where as ${PWD} actually runs print working directory and outputs the result (you can test with a terminal using echo ${PWD} .

For completeness, this stack (based around a BME280 sensor and Ardunio) should be working (for security - I've changed the credentials so if it doesn't work start there!). I've commented it for my own reference, so hope it helps you:

version: '3'

# To Do:
# - Setup defined networks to protect influxdb and telegraf
# - Define a backup process for data
# - Monitor implications of version tags/docker container lifecycles

services: 

    # MQTT Broker, handles data from sensors
    # https://hub.docker.com/_/eclipse-mosquitto
    mosquitto:
        # Name this container so other containers can find it easily
        # Name used in: 
        # - Telegraf config
        container_name: mosquitto
        image: eclipse-mosquitto:1.6
        ports:
            - "1883:1883"
            - "9001:9001"
        depends_on: 
            - influxdb
        restart: always
        volumes:
            # Use a volume for storage
            # since influxdb stores data long term this might not be needed?
            - mosquitto-storage:/mosquitto/data

    # Data storage
    # https://hub.docker.com/_/influxdb
    influxdb:
        # Name this container so other containers can find it easily
        # Name used in: 
        # - Grafana data source 
        # - Telegraf config
        container_name: influxdb
        image: influxdb:1.5
        ports:
            - "8086:8086"
        environment:
            - INFLUXDB_DB=sensors
            - INFLUXDB_ADMIN_USER=admin-user
            - INFLUXDB_ADMIN_PASSWORD=telegraf-admin-password
            - INFLUXDB_USER=telegraf-username
            - INFLUXDB_PASSWORD=telegraf-password
        restart: always
        volumes:
          # Data persistence (could also be a bind mount: /srv/docker/influxdb/data:/var/lib/influxdb)
          - influxdb-storage:/var/lib/influxdb
          # Backups...
          - ./influxdb-backup:/backup
          # Host can run the following on a crontab, then rsnapshot can pickup:
          # docker exec -it influxdb influxd backup -database sensors /backup

    # Message formattet (MQTT -> InfluxDB)
    # https://hub.docker.com/_/telegraf
    telegraf: 
        image: telegraf:1.11
        restart: always
        ports: 
            - "5050:5050"
        depends_on: 
            - influxdb
        volumes:
            # This file needs edited with your MQTT topics, host, etc
            - .docker/telegraf/telegraf.conf:/etc/telegraf/telegraf.conf:ro

    # Dashboard/graphing
    # https://hub.docker.com/r/grafana/grafana
    grafana: 
        # Grafana tags are a mess! just use whatever...
        image: grafana/grafana
        links:
            - influxdb
        hostname: grafana
        ports:
            - "3000:3000"
        depends_on: 
            - influxdb
        volumes:
            # Grafana gets grumpy over bind mount permissions, use a volume
            - grafana-storage:/var/lib/grafana

volumes:
    mosquitto-storage:
    influxdb-storage:
    grafana-storage:

And my config file for telegraf looks like this:

[[inputs.mqtt_consumer]]
  servers = ["tcp://mosquitto:1883"]

  topics = [
    "home/sensor/bme280/temperature",
  ]

  username = "mqttuser"
  password = "mqttpassword"
  data_format = "value"
  data_type = "float"

[[outputs.influxdb]]

  urls = ["http://influxdb:8086"]
  database = "sensors"
  skip_database_creation = true
  timeout = "5s"
  username = "telegraf-username"
  password = "telegraf-password" 
  user_agent = "telegraf"
  udp_payload = "512B"

Note that I am using the container names instead of local IP address within docker.