How to collect all logs from number of servers in docker swarm?

Question

I have number of Linux server that have docker installed on them, all of the server are in a docker swarm, on each server i have a custom application. I also have ELK setup in AWS.

I want to collect all logs from my custom app to the ELK on AWS, I have successfully done that on one server with filebeat by running the following commands:
1. docker pull docker.elastic.co/beats/filebeat-oss:7.3.0
2. created a file in /etc/filebeat/filebeat.yml with the content:

filebeat.inputs:
- type: container
  paths:
  - '/usr/share/continer_logs/*/*.log'
  containers.ids:
  - '111111111111111111111111111111111111111111111111111111111111111111'
  processors:
  - add_docker_metadata: ~
output.elasticsearch:
  hosts: ["XX.YY.ZZ.TT"]

3. chown root:root filebeat.yml
4. sudo docker run -u root -v /etc/filebeat/filebeat.yml:/usr/share/filebeat/filebeat.yml -v /var/lib/docker/containers:/usr/share/continer_logs -v /var/run/docker.sock:/var/run/docker.sock docker.elastic.co/beats/filebeat-oss:7.3.0

And now i want to do the same on all of my docker hosts(And there are a lot of them) in the swarm.

I encounter a number of problems:
1. How do i copy "filebeat.yml" to /etc/filebeat/filebeat.yml on every server?
2. How do i update the "containers.ids" on every server? and how to update it when i upgrade the docker image?

Answer 1

1. How do I copy "filebeat.yml" to /etc/filebeat/filebeat.yml on every server?

You need a configuration management tool for this. I prefer Ansible , you might want to take a look at others.

2. How do I update the "containers.ids" on every server?

You don't have to. Docker manages it by itself IF you use swarm mode. You're using docker run which is meant for the development phase and to deploy applications at one single machine. You need to look at Docker Stack to deploy an application across multiple servers.

How to update it when I upgrade the docker image?

docker stack deploy does both, deploy and update, services.

NOTE: Your image should be present on each node of the swarm in order to get its container deployed on that node.