[英]How do I match a newline in grok/logstash?
I have a remote machine that combines multiline events and sends them across the lumberjack protocol.我有一台远程机器,它结合了多行事件并通过伐木工人协议发送它们。
What comes in is something that looks like this:进来的东西看起来像这样:
{
"message" => "2014-10-20T20:52:56.133+0000 host 2014-10-20 15:52:56,036 [ERROR ][app.logic ] Failed to turn message into JSON\nTraceback (most recent call last):\n File \"somefile.py", line 249, in _get_values\n return r.json()\n File \"/path/to/env/lib/python3.4/site-packages/requests/models.py\", line 793, in json\n return json.loads(self.text, **kwargs)\n File \"/usr/local/lib/python3.4/json/__init__.py\", line 318, in loads\n return _default_decoder.decode(s)\n File \"/usr/local/lib/python3.4/json/decoder.py\", line 343, in decode\n obj, end = self.raw_decode(s, idx=_w(s, 0).end())\n File \"/usr/local/lib/python3.4/json/decoder.py\", line 361, in raw_decode\n raise ValueError(errmsg(\"Expecting value\", s, err.value)) from None\nValueError: Expecting value: line 1 column 1 (char 0), Failed to turn message into JSON"
}
When I try to match the message with当我尝试将消息与
grok {
match => [ "message", "%{TIMESTAMP_ISO8601:timestamp} \[%LOGLEVEL:loglevel}%{ SPACE}\]\[%{NOTSPACE:module}%{SPACE}\]%{GREEDYDATA:message}" ]
}
the GREEDYDATA
is not nearly as greedy as I would like. GREEDYDATA
并不像我希望的那样贪婪。
So then I tried to use gsub:然后我尝试使用 gsub:
mutate {
gsub => ["message", "\n", "LINE_BREAK"]
}
# Grok goes here
mutate {
gsub => ["message", "LINE_BREAK", "\n"]
}
but that one didn't work rather than但那个没有用,而不是
The Quick brown fox
jumps over the lazy
groks
I got我有
The Quick brown fox\njumps over the lazy\ngroks
So...所以...
How do I either add the newline back to my data, make the GREEDYDATA
match my newlines, or in some other way grab the relevant portion of my message?我如何将换行符添加回我的数据,使GREEDYDATA
匹配我的换行符,或者以其他方式获取我的消息的相关部分?
All GREEDYDATA
is is .*
, but .
所有GREEDYDATA
都是.*
,但是.
doesn't match newline, so you can replace %{GREEDYDATA:message}
with (?<message>(.|\\r|\\n)*)
and get it to be truly greedy.不匹配换行符,所以你可以用(?<message>(.|\\r|\\n)*)
替换%{GREEDYDATA:message}
并让它变得真正贪婪。
将正则表达式标志添加到开头允许匹配换行符:
match => [ "message", "(?m)%{TIMESTA...
My final grok for Vertica log using (?m) and [^\\n]+我使用 (?m) 和 [^\\n]+ 对 Vertica 日志的最后理解
match => ["message","(?m)%{TIMESTAMP_ISO8601:ClientTimestamp}%{SPACE}(%{DATA:Action}:)?(%{DATA:ThreadID} )?(\[%{DATA:Module}\] )?(\<%{DATA:Level}\> )?(\[%{DATA:SubAction}\] )?(@%{DATA:Nodename}:)?( (?<Session>(\{.*?\} )?.*?/.*?): )?(?<message>[^\n]+)((\n)?(\t)?(?<StackTrace>[^\n]+))?"]
Thanks to asperla感谢 asperla
https://github.com/elastic/logstash/issues/2282 https://github.com/elastic/logstash/issues/2282
声明:本站的技术帖子网页,遵循CC BY-SA 4.0协议,如果您需要转载,请注明本站网址或者原文地址。任何问题请咨询:yoyou2525@163.com.