[英]Json file from filebeat to Logstash and then to elasticsearch
[英]How to read json file using filebeat and send it to elasticsearch via logstash
这是我的 json 日志文件。 我正在尝试通过我的 logstash 将文件存储到我的弹性搜索中。
{"message":"IM: Orchestration","level":"info"}
{"message":"Investment Management","level":"info"}
这是我的 filebeat.yml
filebeat.inputs:
- type: log
enabled: true
paths:
- D:/Development_Avecto/test-log/tn-logs/im.log
json.keys_under_root: true
json.add_error_key: true
processors:
- decode_json_fields:
fields: ["message"]
output.logstash:
hosts: ["localhost:5044"]
input{
beats {
port => "5044"
}
}
filter {
json {
source => "message"
}
}
output{
elasticsearch{
hosts => ["localhost:9200"]
index => "data"
}
}
无法查看放入elasticserach中的内容。 无法找到错误是什么。 文件节拍日志
2019-06-18T11:30:03.448+0530 INFO registrar/registrar.go:134 从 D:\\Development_Aecto\\filebeat-6.6.2-windows-x86_64\\data\\registry 2018-103.4:134 加载注册商数据+0530 INFO registrar/registrar.go:141 States 从注册商加载:10 2019-06-18T11:30:03.448+0530 WARN beater/filebeat.go:367 Filebeat 无法加载已配置模块的 Ingest Node 管道,因为未配置/启用 Elasticsearch 输出。 如果您已经加载了摄取节点管道或正在使用 Logstash 管道,则可以忽略此警告。 2019-06-18T11:30:03.448+0530 INFO crawler/crawler.go:72 Loading Inputs: 1 2019-06-18T11:30:03.448+0530 INFO log/input.go:138 配置路径:[D:\\Development] \\test-log\\tn-logs\\im.log] 2019-06-18T11:30:03.448+0530 INFO input/input.go:114 开始输入类型:日志; ID:16965758110699470044 2019-06-18T11:30:03.449+0530 INFO crawler/crawler.go:106 加载和启动输入完成。 启用输入:1 2019-06-18T11:30:34.842+0530 INFO [monitoring] log/log.go:144 过去 30 年代的非零指标 {"monitoring": {"metrics": {"beat":{ "cpu":{"system":{"ticks":312,"time":{"ms":312}},"total":{"ticks":390,"time":{"ms":390 },"value":390},"user":{"ticks":78,"time":{"ms":78}}},"handles":{"open":213},"info": {“ephemeral_id”:“66983518-39e6-461c-886d-a1f99da6631d”,“正常运行时间”:{“ms”:30522}},“memstats”:{“gc_next”:4194304,“memory_alloc”:209 :4359488,"rss":22421504}},"filebeat":{"events":{" added":1,"done":1},"harvester":{"open_files":0,"running":0 }},"libbeat":{"config":{"module":{"running":0}},"output":{"type":"logstash"},"pipeline":{"clients":1 ,"events":{"active":0,"filtered":1,"total":1}}},"registrar":{"states":{"current":10,"update":1}, "writes":{"success":1,"total":1}},"system":{"cpu":{"cores":4}}}}} 2
https://www.elastic.co/guide/en/ecs-logging/dotnet/master/setup.html
检查页面底部的第 3 步,了解需要放入 filebeat.yaml 文件的配置:
filebeat.inputs:
- type: log
paths: /path/to/logs.json
json.keys_under_root: true
json.overwrite_keys: true
json.add_error_key: true
json.expand_keys: true
声明:本站的技术帖子网页,遵循CC BY-SA 4.0协议,如果您需要转载,请注明本站网址或者原文地址。任何问题请咨询:yoyou2525@163.com.