Run ELK stack on docker on two server

Question

I want to run ELK on docker, so I run the ELK stack and filebeat containers in one server and projects are running on another server.

how can I transfer the projects log files to the ELK running server?

Answer 1

finally, I found a solution as follows. pleas recommend me if you have any other idea.

1. I run the images of Logstash, Elasticsearch, and Kibana on one server.

Note: I did not run the filebeat image on this server.

2. In the Filebeat Configuration file, we set the output address of the filebeat as below the IP address and port of Logstash that is running on the other server.

    output.logstash: hosts: ["ELK Server IP:5044"]

3. Now I created a compose file for filebeat as follows

   version: '2.2' services: filebeat: build: context: filebeat / volumes: # needed to access all docker logs (read only): - "/var/lib/docker/containers:/usr/share/dockerlogs/data:ro" # needed to access additional information about containers - "/var/run/docker.sock:/var/run/docker.sock" #cpu_count: 1 cpus: 0.5 mem_limit: 2G

4. I run the image of the filebeat by using these settings and compose files on the server where the projects are running.

With these settings, Filebeat can read log files from "/var/lib/docker/containers" and use "hosts: [" ELK Server IP:5044"]" to deliver log files to the Logstash.