[英]Logstash grok filter custom pattern is not working
I've a log file ( http://codepad.org/vAMFhhR2 ), and I want to extract a specific number out of it (line 18) I wrote a custom pattern grok filter, tested it on http://grokdebug.herokuapp.com/ , it works fine and extracts my desired value. 我有一个日志文件( http://codepad.org/vAMFhhR2 ),我想从中提取一个特定的数字(第18行)我写了一个自定义模式grok过滤器,在http:// grokdebug上测试它。 herokuapp.com/ ,它运行正常并提取我想要的值。
here's how logstash.conf looks like: 这是logstash.conf的样子:
input {
tcp {
port => 5000
}
}
filter {
grok{
match => [ "message", "(?<scraped>(?<='item_scraped_count': ).*(?=,))" ]
}
}
output {
elasticsearch {
hosts => "elasticsearch:9200"
}
}
but it doesn't match any record from the same log on Kibana 但它与Kibana上相同日志中的任何记录都不匹配
Thoughts? 思考?
Your regexp may be valid but the lookahead and lookbehind ("?=" and "?<=") are not a good choice in this context. 您的正则表达式可能有效,但前瞻和后瞻(“?=”和“?<=”)在这种情况下不是一个好的选择。 Instead you could use a much simpler filter:
相反,您可以使用更简单的过滤器:
match => [ "message", "'item_scraped_count': %{NUMBER:scraped}" ]
This will extract the number after 'item_scraped_count':
as a field called scraped
, using the 'NUMBER' Grok built-in pattern . 这将解压后的数
'item_scraped_count':
一个名为场scraped
,使用“NUMBER”神交内置模式 。
Result in Kibana: 结果在Kibana:
{
"_index": "logstash-2017.04.11",
"_type": "logs",
"_source": {
"@timestamp": "2017-04-11T20:02:13.194Z",
"scraped": "22",
(...)
}
}
If I may suggest another improvement: since your message is spread across multiple lines you could easily merge it using the multiline
input codec: 如果我可以建议另一个改进:由于您的消息分布在多行中,您可以使用
multiline
输入编解码器轻松合并它:
input {
tcp {
port => 5000
codec => multiline {
pattern => "^(\s|{')"
what => "previous"
}
}
}
This will merge all the lines starting with either a whitespace or {'
with the previous one. 这将合并所有行,以空格开头或
{'
与前一行。
声明:本站的技术帖子网页,遵循CC BY-SA 4.0协议,如果您需要转载,请注明本站网址或者原文地址。任何问题请咨询:yoyou2525@163.com.