[英]Elastic ELK stack 8.5 integration with Spring Boots Application using Filebeat
Setting up a pipeline of elastic search, kibana, and logstash in locally and using filebeat to push logs from a spring boot application to the pipeline.在本地设置弹性搜索、kibana 和 logstash 的管道,并使用 filebeat 将日志从 spring boot 应用程序推送到管道。 U will find the official documentation well-defined, But I created this questions to answer a few points that were not clear.
你会发现官方文档定义明确,但我创建这个问题是为了回答一些不清楚的地方。 I answered for a single spring boot app scenario, thanks to people who are adding their scenarios as well.
我回答了一个单一的 spring boot 应用程序场景,感谢那些也在添加他们的场景的人。
I spend a few days configuring the ELK stack with my spring boot application.我花了几天时间用我的 spring boot 应用程序配置 ELK 堆栈。 Here I won't specify the step-by-step integration, for that, you can refer to the official documentation.
这里就不详细说明具体的步骤了,具体可以参考官方文档。 This is more focused on what I didn't find in the documentation steps.
这更侧重于我在文档步骤中没有找到的内容。
Env: This will be focused on setting up the 8.5.3 version in a mac os. Env:这将着重于在 mac os 中设置 8.5.3 版本。
For Elasticsearch and Kibana I didn't have any trouble following the official document word by word.对于 Elasticsearch 和 Kibana,我逐字逐句地阅读官方文档没有遇到任何问题。
Elasticsearch: https://www.elastic.co/downloads/elasticsearch弹性搜索: https ://www.elastic.co/downloads/elasticsearch
Kibana: https://www.elastic.co/downloads/kibana基巴纳: https ://www.elastic.co/downloads/kibana
In my project, I needed to extract only a specific log line and process it.在我的项目中,我只需要提取特定的日志行并对其进行处理。 U can use the below official document link to download and extract the logstash and filebeat.
您可以使用下面的官方文档链接下载并提取 logstash 和 filebeat。 Then you can use the mentioned configs before you run it.
然后你可以在运行它之前使用提到的配置。
Logstash: https://www.elastic.co/downloads/logstash Logstash: https ://www.elastic.co/downloads/logstash
Filebeat: https://www.elastic.co/downloads/beats/filebeat文件拍: https ://www.elastic.co/downloads/beats/filebeat
Filebeat:文件拍:
First, you need to make permission changes to your filebeat.yml file.首先,您需要更改 filebeat.yml 文件的权限。 Navigate to your filebeat extracted folder and you can use the following config if needed.
导航到您的 filebeat 提取文件夹,如果需要,您可以使用以下配置。
filebeat.inputs:
- type: filestream
id: filebeat-id-name
enabled: true
paths:
- /Users/leons/IdeaProjects/SpringELKDemo/myapplogs.log #Path to you log file
#I wanted to only read the log line with MainController string in it
include_lines: ['MainController']
output.logstash:
hosts: ["localhost:5044"]
Then you need to alter the write permission for this file using the below command(mac).然后您需要使用以下命令 (mac) 更改此文件的写入权限。 Later you can edit the file using sudo nano.
稍后您可以使用 sudo nano 编辑该文件。
sudo chown root filebeat.yml
Logstash:日志存储:
Initial a sample file for logstash.conf is available in the config folder inside logstash. logstash.conf 的初始示例文件位于 logstash 的配置文件夹中。 you can refer to that, also take a look at mine.
你可以参考那个,也看看我的。
input {
beats {
port => 5044
}
}
filter {
dissect {
mapping => {
"message" => "%{}: %{data_message}"
}
}
json {
source => "data_message"
}
}
output {
elasticsearch {
hosts => ["https://localhost:9200"]
index => "index_name"
user => "elastic"
password => "XXXXXXXXXXXXX-XXX"
ssl_certificate_verification => false
}
stdout{
codec => rubydebug
}
}
I used the dissect filter to do string manipulation in my logline, that filebeat transferred.我使用 dissect 过滤器在我的日志行中进行字符串操作,即 filebeat 传输。 Below was my log, and I needed only the exact message which is JSON string
下面是我的日志,我只需要确切的消息,即 JSON 字符串
2022-12-15 21:14:56.152 INFO 9278 --- [http-nio-8080-exec-10] c.p.t.springdemo.controller.MainController : {"name":"leons","id":"123123","msg":"hello world"}
For more on dissect refer official docs有关解剖的更多信息,请参阅官方文档
The json filter is used to convert the JSON key: values into fields and values in your elastic document. json 过滤器用于将 JSON key: values 转换为弹性文档中的字段和值。
Now you should be ready to run logstash and filebeat using official document command.现在您应该准备好使用官方文档命令运行 logstash 和 filebeat。 Just for reference use below
下面仅供参考使用
Logstash:日志存储:
bin/logstash -f logstash.conf
Filebeat:文件拍:
sudo ./filebeat -e -c filebeat.yml
声明:本站的技术帖子网页,遵循CC BY-SA 4.0协议,如果您需要转载,请注明本站网址或者原文地址。任何问题请咨询:yoyou2525@163.com.