Grok模式以匹配具有多个特殊字符的日志

Question

我想用ELK捕获我的异常，但是我的异常充满了( { [ . , \\ / , " '字符。如何在grok中将它们编入索引？

我的日志文件：

Exception in *** CoreLevel*** occured. 
Date&Time: 2018-01-21 09:52:20.744092 
Root: 
 ( ['MQROOT' : 0x7f0a902b2d80]
  (0x01000000:Name  ):Properties = ( ['MQPROPERTYPARSER' : 0x7f0a902bffa0]
    (0x03000000:NameValue):MessageFormat          = 'jms_text' (CHARACTER)  )
    (0x03000000:NameValue):MsgId            = X'5059414d313339363131303234383030303238' (BLOB))
    (0x01000000:Name     ):usr            = (
      (0x03000000:NameValue):MessageName = 'SampleMessageName' (CHARACTER)
      (0x03000000:NameValue):MsgVersion  = 'V1' (CHARACTER)
    )
  )
) 
***************************************************************************************** 
***************************************************************************************** 
ExceptionList:  
( ['MQROOT' : 0x7f0a9072b350]
  (0x01000000:Name):RecoverableException = (
    (0x03000000:NameValue):File                 = '/build/slot1/S800_P/src/DataFlowEngine/PluginInterface/ImbJniNode.cpp' (CHARACTER)
    (0x03000000:NameValue):Line                 = 1260 (INTEGER)
    (0x03000000:NameValue):Text                 = 'Caught exception and rethrowing' (CHARACTER)
    (0x01000000:Name     ):Insert               = (
      (0x03000000:NameValue):Type = 14 (INTEGER)
    )
          (0x03000000:NameValue):Label    = '' (CHARACTER)
          (0x03000000:NameValue):Catalog  = "BIPmsgs" (CHARACTER)
          (0x03000000:NameValue):Severity = 3 (INTEGER)
          (0x03000000:NameValue):Number   = 4395 (INTEGER)
   )
  )
)

我除了把这种模式带到基巴那

Exception in: CoreLevel, 
Date&Time: 2018-01-21 09:52:20.744092
message:{
  Root:".....",
  ExceptionList:"......"
}

这是我的骗子块，不起作用

 grok {
    patterns_dir => "/etc/logstash/patterns/"
    break_on_match => false
    keep_empty_captures => true

    match => {"message" => ["Exception in (?<msg_f> occured..) Date&Time: %{SYSLOGTIMESTAMP:timestamp}"]}
 }

  mutate {
        gsub => ["message", "\n", ""]
 }

如果有人可以帮助我，我将不胜感激。

Answer 1

日志中的日期采用ISO8601格式，因此可以与TIMESTAMP_ISO8601预定义模式匹配。 对于日期和时间之后的行，可以使用(?m)将日志中的多行与GREEDYDATA 。

以下模式会起作用，

Exception in \*\*\* %{WORD:Exception_in}.*\s*Date&Time: %{TIMESTAMP_ISO8601}(?m)%{GREEDYDATA}

它将输出，

{
  "Exception_in": [
    [
      "CoreLevel"
    ]
  ],
  "TIMESTAMP_ISO8601": [
    [
      "2018-01-21 09:52:20.744092"
    ]
  ],
  "YEAR": [
    [
      "2018"
    ]
  ],
  "MONTHNUM": [
    [
      "01"
    ]
  ],
  "MONTHDAY": [
    [
      "21"
    ]
  ],
  "HOUR": [
    [
      "09",
      null
    ]
  ],
  "MINUTE": [
    [
      "52",
      null
    ]
  ],
  "SECOND": [
    [
      "20.744092"
    ]
  ],
  "ISO8601_TIMEZONE": [
    [
      null
    ]
  ],
  "GREEDYDATA": [
    [
      " \nRoot: \n ( ['MQROOT' : 0x7f0a902b2d80]\n  (0x01000000:Name  ):Properties = ( ['MQPROPERTYPARSER' : 0x7f0a902bffa0]\n    (0x03000000:NameValue):MessageFormat          = 'jms_text' (CHARACTER)  )\n    (0x03000000:NameValue):MsgId            = X'5059414d313339363131303234383030303238' (BLOB))\n    (0x01000000:Name     ):usr            = (\n      (0x03000000:NameValue):MessageName = 'SampleMessageName' (CHARACTER)\n      (0x03000000:NameValue):MsgVersion  = 'V1' (CHARACTER)\n    )\n  )\n) \n***************************************************************************************** \n***************************************************************************************** \nExceptionList:  \n( ['MQROOT' : 0x7f0a9072b350]\n  (0x01000000:Name):RecoverableException = (\n    (0x03000000:NameValue):File                 = '/build/slot1/S800_P/src/DataFlowEngine/PluginInterface/ImbJniNode.cpp' (CHARACTER)\n    (0x03000000:NameValue):Line                 = 1260 (INTEGER)\n    (0x03000000:NameValue):Text                 = 'Caught exception and rethrowing' (CHARACTER)\n    (0x01000000:Name     ):Insert               = (\n      (0x03000000:NameValue):Type = 14 (INTEGER)\n    )\n          (0x03000000:NameValue):Label    = '' (CHARACTER)\n          (0x03000000:NameValue):Catalog  = "BIPmsgs" (CHARACTER)\n          (0x03000000:NameValue):Severity = 3 (INTEGER)\n          (0x03000000:NameValue):Number   = 4395 (INTEGER)\n   )\n  )\n)"
    ]
  ]
}

你可以在这里测试