[英]Parsing JSON file into logstash
Hi I am trying to send a json file with multiple objects to elasticsearch with the logstash so I can display the data using kibana. 嗨,我正尝试使用logstash将包含多个对象的json文件发送到elasticsearch,以便我可以使用kibana显示数据。 I have researched this extensively and simply cannot understand how to make the data formatted correctly to be used in kibana.
我对此进行了广泛的研究,根本无法理解如何正确格式化数据以在kibana中使用。
I have tried to use different filters such as: json, date, and grok 我尝试使用不同的过滤器,例如:json,date和grok
The issue is probably how I'm going about using these filters as I can't understand it's setup all to well. 问题可能是我如何使用这些过滤器,因为我不明白它的设置是否很好。
Here is a sample line of the input json file: 这是输入json文件的示例行:
{"time":"2015-09-20;12:13:24","bug_code":"tr","stacktrace":"543534"},
I want to use this format for displaying the data in kibana and sorting many objects according to their "time" 我想使用这种格式在kibana中显示数据并根据其“时间”对许多对象进行排序
this following is what my current filter section is: 以下是我当前的过滤器部分:
filter {
date {
match => ["time", "YYYY-MM-dd;HH:mm:ss Z" ]
timezone => "America/New_York"
locale => "en"
target => "@timestamp"
}
grok {
match => ["time", "%{TIMESTAMP_ISO8601:timestamp}"]
}
}
At this point I know the grok is wrong because I get "_grokparsefailure" but how can I figure out the correct way to use grok or is there a simple way to sort the data using the given timestamp and not the processed timestamp given when sending the data through. 在这一点上,我知道grok是错误的,因为我得到了“ _grokparsefailure”,但是我如何找出使用grok的正确方法,还是有一种简单的方法来使用给定的时间戳对数据进行排序,而不是发送时所给的处理后的时间戳数据通过。
here is what the output currently shows: 这是当前输出显示的内容:
"message" => "{\"time\":\"2015-09-20;12:13:24\",\"bug_code\":\"tr\",\"stacktrace\":\"543534\"},\r",
"@version" => "1",
"@timestamp" => "2015-11-23T09:54:50:274Z",
"host" => "<my_computer>",
"path" => "<path_to_.json>",
"type" => "json",
"tags" => [
[0] "_grokparsefailure"
any advice would be very much appreciated 任何建议将不胜感激
You're almost there, I could get it working with a few tweaks. 您快到了,我可以通过一些调整使其工作。
First, you need to add the json{}
filter in the first position. 首先,您需要在第一个位置添加
json{}
过滤器。 Then you need to change the date pattern to YYYY-MM-dd;HH:mm:ss
and finally you can remove the grok
filter at the end. 然后,您需要将日期模式更改为
YYYY-MM-dd;HH:mm:ss
,最后可以在最后删除grok
过滤器。 You filter configuration would look like this: 您的过滤器配置如下所示:
filter {
json {
source => "message"
}
date {
match => ["time", "YYYY-MM-dd;HH:mm:ss" ]
timezone => "America/New_York"
locale => "en"
target => "@timestamp"
}
}
The parsed event for your sample JSON line would then look like this: 然后,示例JSON行的已解析事件将如下所示:
{
"message" => "{\"time\":\"2015-09-20;12:13:24\",\"bug_code\":\"tr\",\"stacktrace\":\"543534\"}",
"@version" => "1",
"@timestamp" => "2015-09-20T16:13:24.000Z",
"host" => "iMac.local",
"time" => "2015-09-20;12:13:24",
"bug_code" => "tr",
"stacktrace" => "543534"
}
声明:本站的技术帖子网页,遵循CC BY-SA 4.0协议,如果您需要转载,请注明本站网址或者原文地址。任何问题请咨询:yoyou2525@163.com.