[英]Grok patterns to match log with multiple special characters
I want to catch my exception with ELK but my exception is full of ( { [ . , \\ / , " '
character. How can I index them in grok? 我想用ELK捕获我的异常,但是我的异常充满了( { [ . , \\ / , " '
字符。如何在grok中将它们编入索引?
My log file: 我的日志文件:
Exception in *** CoreLevel*** occured.
Date&Time: 2018-01-21 09:52:20.744092
Root:
( ['MQROOT' : 0x7f0a902b2d80]
(0x01000000:Name ):Properties = ( ['MQPROPERTYPARSER' : 0x7f0a902bffa0]
(0x03000000:NameValue):MessageFormat = 'jms_text' (CHARACTER) )
(0x03000000:NameValue):MsgId = X'5059414d313339363131303234383030303238' (BLOB))
(0x01000000:Name ):usr = (
(0x03000000:NameValue):MessageName = 'SampleMessageName' (CHARACTER)
(0x03000000:NameValue):MsgVersion = 'V1' (CHARACTER)
)
)
)
*****************************************************************************************
*****************************************************************************************
ExceptionList:
( ['MQROOT' : 0x7f0a9072b350]
(0x01000000:Name):RecoverableException = (
(0x03000000:NameValue):File = '/build/slot1/S800_P/src/DataFlowEngine/PluginInterface/ImbJniNode.cpp' (CHARACTER)
(0x03000000:NameValue):Line = 1260 (INTEGER)
(0x03000000:NameValue):Text = 'Caught exception and rethrowing' (CHARACTER)
(0x01000000:Name ):Insert = (
(0x03000000:NameValue):Type = 14 (INTEGER)
)
(0x03000000:NameValue):Label = '' (CHARACTER)
(0x03000000:NameValue):Catalog = "BIPmsgs" (CHARACTER)
(0x03000000:NameValue):Severity = 3 (INTEGER)
(0x03000000:NameValue):Number = 4395 (INTEGER)
)
)
)
and I except to get this pattern into kibana 我除了把这种模式带到基巴那
Exception in: CoreLevel,
Date&Time: 2018-01-21 09:52:20.744092
message:{
Root:".....",
ExceptionList:"......"
}
and this is my grok block that doesn't work 这是我的骗子块,不起作用
grok {
patterns_dir => "/etc/logstash/patterns/"
break_on_match => false
keep_empty_captures => true
match => {"message" => ["Exception in (?<msg_f> occured..) Date&Time: %{SYSLOGTIMESTAMP:timestamp}"]}
}
mutate {
gsub => ["message", "\n", ""]
}
I'd really appreciate if anyone could help me. 如果有人可以帮助我,我将不胜感激。
The date in your log is in ISO8601 format so it can be matched with TIMESTAMP_ISO8601
predefined pattern. 日志中的日期采用ISO8601格式,因此可以与TIMESTAMP_ISO8601
预定义模式匹配。 For lines after date & time, you can use (?m)
to match multiline in your log with GREEDYDATA
. 对于日期和时间之后的行,可以使用(?m)
将日志中的多行与GREEDYDATA
。
Following pattern will work, 以下模式会起作用,
Exception in \*\*\* %{WORD:Exception_in}.*\s*Date&Time: %{TIMESTAMP_ISO8601}(?m)%{GREEDYDATA}
It will output, 它将输出,
{
"Exception_in": [
[
"CoreLevel"
]
],
"TIMESTAMP_ISO8601": [
[
"2018-01-21 09:52:20.744092"
]
],
"YEAR": [
[
"2018"
]
],
"MONTHNUM": [
[
"01"
]
],
"MONTHDAY": [
[
"21"
]
],
"HOUR": [
[
"09",
null
]
],
"MINUTE": [
[
"52",
null
]
],
"SECOND": [
[
"20.744092"
]
],
"ISO8601_TIMEZONE": [
[
null
]
],
"GREEDYDATA": [
[
" \nRoot: \n ( ['MQROOT' : 0x7f0a902b2d80]\n (0x01000000:Name ):Properties = ( ['MQPROPERTYPARSER' : 0x7f0a902bffa0]\n (0x03000000:NameValue):MessageFormat = 'jms_text' (CHARACTER) )\n (0x03000000:NameValue):MsgId = X'5059414d313339363131303234383030303238' (BLOB))\n (0x01000000:Name ):usr = (\n (0x03000000:NameValue):MessageName = 'SampleMessageName' (CHARACTER)\n (0x03000000:NameValue):MsgVersion = 'V1' (CHARACTER)\n )\n )\n) \n***************************************************************************************** \n***************************************************************************************** \nExceptionList: \n( ['MQROOT' : 0x7f0a9072b350]\n (0x01000000:Name):RecoverableException = (\n (0x03000000:NameValue):File = '/build/slot1/S800_P/src/DataFlowEngine/PluginInterface/ImbJniNode.cpp' (CHARACTER)\n (0x03000000:NameValue):Line = 1260 (INTEGER)\n (0x03000000:NameValue):Text = 'Caught exception and rethrowing' (CHARACTER)\n (0x01000000:Name ):Insert = (\n (0x03000000:NameValue):Type = 14 (INTEGER)\n )\n (0x03000000:NameValue):Label = '' (CHARACTER)\n (0x03000000:NameValue):Catalog = "BIPmsgs" (CHARACTER)\n (0x03000000:NameValue):Severity = 3 (INTEGER)\n (0x03000000:NameValue):Number = 4395 (INTEGER)\n )\n )\n)"
]
]
}
You can test it here 你可以在这里测试
声明:本站的技术帖子网页,遵循CC BY-SA 4.0协议,如果您需要转载,请注明本站网址或者原文地址。任何问题请咨询:yoyou2525@163.com.