[英]Filebeat container does not send logs to Elastic
On my local machine running Ubuntu 18.04 via "Windows Subsystem Linux 2" on Windows 10, I am running Elastic 7.3, Kibana 7.3 and Elastic 7.3 docker containers.在通过“Windows 子系统 Linux 2”在 Windows 10 上运行 Ubuntu 18.04 的本地计算机上,我正在运行 Elastic 7.3、Kibana 7.3 和 Elastic 7.3 docker 容器。
Set-up is successful and Filebeat seems to monitor containers correctly.设置成功,Filebeat 似乎可以正确监控容器。 However, Kibana does not show any logs.但是,Kibana 不显示任何日志。
Setup设置
To set-up Elastic and Kibana I use the following commands要设置 Elastic 和 Kibana,我使用以下命令
docker pull docker.elastic.co/elasticsearch/elasticsearch:7.3.1
docker run --network=lognetwork --name=elasticsearch -p 9200:9200 -p 9300:9300 -e "discovery.type=single-node" docker.elastic.co/elasticsearch/elasticsearch:7.3.1
docker pull docker.elastic.co/kibana/kibana:7.3.1
docker run --name=kibana --network=lognetwork -e ELASTICSEARCH_HOSTS=http://elasticsearch:9200 -p 5601:5601 docker.elastic.co/kibana/kibana:7.3.1
After these two commands, the logs from Kibana container show it successfully connects to Elastic:在这两个命令之后,来自 Kibana 容器的日志显示它成功连接到 Elastic:
{"type":"log","@timestamp":"2019-09-01T13:22:18Z","tags":["status","plugin:spaces@7.3.1","info"],"pid":6,"state":"green","message":"Status changed from yellow to green - Ready","prevState":"yellow","prevMsg":"Waiting for Elasticsearch"}
I can also go to Kibana dashboard on http://localhost:5601 as well as Elastic on http://localhost:9200 both function properly我也可以 go 到http://localhost:5601上的 Kibana 仪表板以及http://localhost:9200上的 Elastic 两个 function
I then set up filebeat:然后我设置了 filebeat:
docker run --network=lognetwork docker.elastic.co/beats/filebeat:7.3.1 setup -E setup.kibana.host=kibana:5601 -E output.elasticsearch.hosts=["elasticsearch:9200"]
I can see both Elastic and Kibana container logs and returning 200. The logs on the Filebeat container show:我可以看到 Elastic 和 Kibana 容器日志并返回 200。Filebeat 容器上的日志显示:
Index setup finished.
Loading dashboards (Kibana must be running and reachable)
Loaded dashboards
Loaded machine learning job configurations
Loaded Ingest pipelines
Finally, I pull the default config from Elastic site, launch Filebeat and attach to the container最后,我从 Elastic 站点拉取默认配置,启动 Filebeat 并附加到容器
curl -L -O https://raw.githubusercontent.com/elastic/beats/7.3/deploy/docker/filebeat.docker.yml
docker run -d --network=lognetwork --name=filebeat --user=root --volume="$(pwd)/filebeat.docker.yml:/usr/share/filebeat/filebeat.yml:ro" --volume="/var/lib/docker/containers:/var/lib/docker/containers:ro" --volume="/var/run/docker.sock:/var/run/docker.sock:ro" docker.elastic.co/beats/filebeat:7.3.1 filebeat -e -strict.perms=false -E output.elasticsearch.hosts=["elasticsearch:9200"]
docker attach filebeat
I can see Filebeat sending monitoring pulse but when it does, elastic logs do not show anything new.我可以看到 Filebeat 发送监控脉冲,但当它发送时,弹性日志不会显示任何新内容。
To test, I launch Docker "hello-world" which generates several lines of logs为了测试,我启动了 Docker “hello-world”,它生成了几行日志
docker run hello-world
Filebeat shows the following log Filebeat 显示如下日志
2019-09-01T13:30:40.624Z INFO log/input.go:148 Configured paths: [/var/lib/docker/containers/460cc8c215ff69ecf28685c9cf89c0e56d0b3e4f680b8bf29beb5b570ebb7a14/*-json.log]
2019-09-01T13:30:40.624Z INFO input/input.go:114 Starting input of type: container; ID: 16402101064670842079
I then go to http://localhost:5601我然后 go 到http://localhost:5601
Results :结果:
Kibana shows no logs. Kibana 不显示任何日志。 Clicking for "check for new data" does not show anything either.单击“检查新数据”也不会显示任何内容。
The folder /var/lib/docker/containers is also empty.文件夹 /var/lib/docker/containers 也是空的。 The path returned by filebeat log (/var/lib/docker/containers/460cc8c215ff69ecf28685c9cf89c0e56d0b3e4f680b8bf29beb5b570ebb7a14/) does not seem to exist. filebeat log返回的路径(/var/lib/docker/containers/460cc8c215ff69ecf28685c9cf89c0e56d0b3e4f680b8bf29beb5b570ebb7a14/)好像不存在。
Expected :预期:
- Kibana to show the "hello world" docker container logs - Kibana 显示“hello world”docker 容器日志
- To see a log file under /var/lib/docker/containers - 查看 /var/lib/docker/containers 下的日志文件
What am I missing?我错过了什么?
Thank you,谢谢你,
Olivier奥利维尔
Well, it took me many hours before asking on SO, and of course, 30mn after asking I found the answer. 好吧,我花了很多小时才提出SO,当然,问了我之后才找到答案了3000万。
The trick was to check where the logs were created as running Docker-Desktop on WSL2 is slightly different than running Docker on Linux. 诀窍是检查日志的创建位置,因为在WSL2上运行Docker-Desktop与在Linux上运行Docker略有不同。
docker inspect filebeat | grep LogPath
returns: 收益:
"LogPath": "/var/data/docker-desktop/default/daemon-data/containers/fd56c5e43c9206baaadd33d3a711e523107622450d0deafb498e7940d809f779/fd56c5e43c9206baaadd33d3a711e523107622450d0deafb498e7940d809f779-json.log
Then changing the volume map accordingly volume="/var/data/docker-desktop/default/daemon-data/containers:/var/lib/docker/containers:ro" when launching filebeat did the job: 然后在启动filebeat时相应地更改卷映射volume =“ / var / data / docker-desktop / default / daemon-data / containers:/ var / lib / docker / containers:ro” :
docker run -d --network=lognetwork --name=filebeat --user=root --volume="$(pwd)/filebeat.docker.yml:/usr/share/filebeat/filebeat.yml:ro" --volume="/var/data/docker-desktop/default/daemon-data/containers:/var/lib/docker/containers:ro" --volume="/var/run/docker.sock:/var/run/docker.sock:ro" docker.elastic.co/beats/filebeat:7.3.1 filebeat -e -strict.perms=false -E output.elasticsearch.hosts=["elasticsearch:9200"]
The logs are now properly shown on kibana 现在,日志已正确显示在kibana上
In my case:就我而言:
Docker desktop installed in Windows 10 + WSL2 enabled in docker. Docker 桌面安装在 Windows 10 + WSL2 在 docker 中启用。
The pipeline: Filebeat -> logstash -> elastic search -> kibana
管道: Filebeat -> logstash -> elastic search -> kibana
Problem: Filebeat was not finding logs from docker. But from a local mounted folder it was sending logs to ELK and was showing up in kibana.问题:Filebeat 没有找到来自 docker 的日志。但是它从本地安装的文件夹向 ELK 发送日志并显示在 kibana 中。
Solution: I was running docker-compose up from wsl bash shell. Instead I ran the same from windows powershell, or cmd and the logs from docker containers started to appear in kibana.解决方案:我从 wsl bash shell 运行 docker-compose。相反,我从 windows powershell 或 cmd 运行相同的日志,并且来自 884065722228588 容器的日志开始出现在 kanaib 容器中
In docker-compose
file:在docker-compose
文件中:
filebeat:
volumes:
- /var/run/docker.sock:/var/run/docker.sock
- /var/lib/docker:/var/lib/docker
- ./MYLOG_TEST:/usr/share/filebeat/mylog
- ./MY_filebeat.yml:/usr/share/filebeat/filebeat.yml
and in MY_filebeat.yml
:在MY_filebeat.yml
中:
filebeat.inputs:
#for docker logs
- type: container # for older filestream version use docker as type
enabled: true
paths:
- /var/lib/docker/containers/**/*.log
#for my test log files
- type: log # for filebeat latest versions8.1+, use filestream as type
enabled: true
paths:
- /usr/share/filebeat/mylog/*.log
声明:本站的技术帖子网页,遵循CC BY-SA 4.0协议,如果您需要转载,请注明本站网址或者原文地址。任何问题请咨询:yoyou2525@163.com.