How to create rolling logs for Filebeat within a docker container

Question

I'm new to log4j2 and the elastic stack.

I have a filebeat docker container that doesn't work exactly how I want and now I want to take a look at the logs. But when I do docker-compose logs I get a lot debug messages and json objects. It's unreadable how much there is.

How can I create a log4j2 properties setup to create some rolling log files. Maybe put the old logs into a monthly based folder or something? and where do I put this log4j2.properties file?

Answer 1

It's generating a lot of logs because you're running docker-compose logs , which will get the logs for all containers in your docker compose file.

What you want is probably:

• docker logs <name-of-filebeat-container> . The name of the filebeat container can be found doing a docker ps .
• docker compose logs <name-of-filebeat-service> . The name of the service can be found on your docker-composer.yml file.

Regarding the JSON outputs, you can query your Docker engine default logging driver with:

# docker info | grep 'Logging Driver'
Logging Driver: json-file

If your container have a different Logging Driver you can check with:

docker inspect -f '{{.HostConfig.LogConfig.Type}}' <name-or-id-of-the-container>

You can find all log drivers in this link

To run containers with a different log-driver you can do:

• With docker run: docker run -it --log-driver <log-driver> alpine ash

• With docker-compose:

   `logging: driver: syslog options: syslog-address: "tcp://192.168.0.42:123"` 

Regarding your log rotation questio, I'd say the easyest way is to configure the logging driver with the syslog driver, configure it to your local machine (or your syslog server) and then logrotate the files.

You can find several logrotate articles for Linux (which I assume you're using), for example this one