Mac OS X, Docker: Get container logs out of Docker for Mac VM into an ELK Stack in the same Docker for Mac

Question

Problem:
- ELK Stack (7.6.2) running in Docker for Mac (2.2.0.5)
- Learned from Docker container log file not found on Mac that container logs on Mac are kept in the Docker for Mac VM.

Question:
How can I get (some of) the container logs - continuously - out of the VM into the ELK Stack which runs on the same Docker for Mac.

Yes, I know in Linux it would be much easier. But currently I only have my Mac.
Yes, I know I could copy the files with some Mac magic out of the VM into the normal Mac FS and then throw it into the ELK stack. But I want to avoid this manual step, if possible. Yes I know I could make a cron job, which does it automatically, but I want to avoid that, too.

So any idea how to achieve this?

Thanks, Alex

Answer 1

Ok, after more research and some try and error, here is my solution, which may not be the best, but it is working.

Steps:
1) Create network:

docker network create elastic  

2) Run es:

docker run -d --name elasticsearch  --network elastic --restart unless-stopped -v /Data/elastic:/usr/share/elasticsearch/data -p 9200:9200 -p 9300:9300 -e "discovery.type=single-node" elasticsearch:7.6.2  

3) Run kibana:

docker run -d --name kibana --network elastic --restart unless-stopped -p 5601:5601 kibana:7.6.2

4) Create a logstash.conf:

input {
  syslog {
    port => 9500
    type => "docker"
  }
}

filter {
  date {
    match => [ "timestamp" , "dd/MMM/yyyy:HH:mm:ss Z" ]
  }
}

output {
  elasticsearch {
    hosts => ["elasticsearch:9200"]
    index => "docker_logs"
  }
}

5) Run logstash:

docker run -d --rm -v $PWD/config/syslog.conf:/usr/share/logstash/config/logstash.conf -p 9500:9500 --name logstash --network elastic logstash:7.6.2 bin/logstash -f /usr/share/logstash/config/logstash.conf

6) Run container:

docker run --log-driver syslog --log-opt syslog-address=tcp://1.2.3.4:9500 alpine echo hello world

Being 1.2.3.4 the ip of the docker host.

7) Open kibana:

open http://1.2.3.4:5601

8) Do the kibana stuff I wanted to learn

Ok, that's my solution...