Apache Catalina日志的Grok模式

Question

我有一些Apache Catalina日志，看起来像这样：

[22/Jul/2016:09:22:37 +0000] 10.10.29.1 - GET GET /static/s/en/providerLayer_ROOT.js HTTP/1.1  200 6298 HTTP/1.1 Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/51.0.2704.103 Safari/537.36 https://wpqa.test.com/app/prov/provSelectAccount.htm
[22/Jul/2016:09:22:37 +0000] 10.10.29.1 - GET GET /static/s/en/gregorian.js HTTP/1.1  200 4987 HTTP/1.1 Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/51.0.2704.103 Safari/537.36 https://wpqa.test.com/app/prov/provSelectAccount.htm

我该如何写一个grok模式来匹配这些。 我在网上四处张望，但找不到太多东西。 我试过了：

match => [ "message", "%{TOMCATLOG}", "message", "%{CATALINALOG}" ]  

但是想获得更多细节。

[22/Jul/2016:09:22:37 +0000] --is date time
10.10.29.1                   --is Ip address
GET GET                      --HTTP Method
/static/s/en/providerLayer_ROOT.js -- Request
HTTP/1.1                     --Protocol Version
200                          --HTTP Status
6298                         --Response time
HTTP/1.1 Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/51.0.2704.103 Safari/537.36  --Browser info
https://wpqa.test.com/app/prov/provSelectAccount.htm -- Called URL

试图突破这一点已经使我完全迷失了，无论我不断得到_grokparsefailures的正则表达式。 我的特征码文件中缺少什么吗？

谢谢，

Answer 1

我使用了以下grok过滤器，它非常适合您的日志：

%{SYSLOG5424SD:timestamp} %{IPV4:IP} - %{CRON_ACTION:HTTPMETHOD}%{URIPATH:request} %{NOTSPACE:protocolVersion}  %{NUMBER:status} %{NUMBER:responseTime} %{NOTSPACE:browserinfo} %{NOTSPACE:browserinfo} (?<browserinfo>(\((.*)\))) %{NOTSPACE:browserinfo} %{NOTSPACE:browserinfo} %{URI:calledURL}

这是输出：

{
  "timestamp": [
    [
      "[22/Jul/2016:09:22:37 +0000]"
    ]
  ],
  "IP": [
    [
      "10.10.29.1"
    ]
  ],
  "HTTPMETHOD": [
    [
      "GET GET "
    ]
  ],
  "request": [
    [
      "/static/s/en/gregorian.js"
    ]
  ],
  "protocolVersion": [
    [
      "HTTP/1.1"
    ]
  ],
  "status": [
    [
      "200"
    ]
  ],
  "responseTime": [
    [
      "4987"
    ]
  ],
  "browserinfo": [
    [
      "HTTP/1.1",
      "Mozilla/5.0",
      "Chrome/51.0.2704.103",
      "Safari/537.36"
    ],
    [
      "(Windows NT 6.1; WOW64) AppleWebKit/537.36 (KHTML, like Gecko)"
    ]
  ],
  "calledURL": [
    [
      "https://wpqa.test.com/app/prov/provSelectAccount.htm"
    ]
  ]
}

您可以使用grok调试器在此处调试grok过滤器
您可以在此链接上使用grok构造函数