繁体   English   中英

Logstash不将过滤器应用于Apache日志

[英]Logstash not applying filter to Apache logs

我正在尝试使用ELK堆栈来解析一些Apache访问日志,但是Logstash没有应用我在任何Apache日志上创建的Apache过滤器时遇到问题。 这是我的过滤器文件:

filter {
  if [type] == "apache_access" {
    grok {
      patterns_dir => ["/opt/logstash/patterns/apache"]
      add_tag => ["grokked", "apache"]
      match => ["messege", "%{IP:client} - - \[%{HTTPDATE:event_date}\] %{QS:first} %{NUMBER:response} %{NUMBER:bytes} %{QS:destination} %{QS:browser}"]
    }
  }
}

filebeat配置:

filebeat:
  prospectors:
    -
      paths:
        - /var/log/apache2/access.log
      document_type: apache_access
  registry_file: /var/lib/filebeat/registry

另外,我使用的是logz.io中的示例日志文件,其中包含类似以下的日志:

88.114.162.149 - - [04/Aug/2016:00:00:05 +0000] "GET /item/giftcards/3802 HTTP/1.1" 200 82 "/category/books" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10.6; rv:9.0.1) Gecko/20100101 Firefox/9.0.1"
156.141.192.36 - - [04/Aug/2016:00:00:10 +0000] "GET /category/toys?from=20 HTTP/1.1" 200 135 "/category/toys" "Mozilla/5.0 (Windows NT 6.0) AppleWebKit/535.11 (KHTML, like Gecko) Chrome/17.0.963.56 Safari/535.11"
92.213.110.215 - - [04/Aug/2016:00:00:15 +0000] "GET /category/software HTTP/1.1" 200 108 "/category/books" "Mozilla/5.0 (Windows NT 6.0) AppleWebKit/535.11 (KHTML, like Gecko) Chrome/17.0.963.56 Safari/535.11"
80.225.119.24 - - [04/Aug/2016:00:00:20 +0000] "GET /category/cameras HTTP/1.1" 200 100 "http://www.google.com/search?ie=UTF-8&q=google&sclient=psy-ab&q=Cameras+Books&oq=Cameras+Books&aq=f&aqi=g-vL1&aql=&pbx=1&bav=on.2,or.r_gc.r_pw.r_qf.,cf.osb&biw=2640&bih=427" "Mozilla/4.0 (compatible; MSIE 8.0; Windows NT 6.1; WOW64; Trident/4.0; YTB730; GTB7.2; SLCC2; .NET CLR 2.0.50727; .NET CLR 3.5.30729; .NET CLR 3.0.30729; .NET4.0C; .NET4.0E; Media Center PC 6.0)"
208.219.150.176 - - [04/Aug/2016:00:00:25 +0000] "GET /category/software HTTP/1.1" 200 117 "-" "Mozilla/4.0 (compatible; MSIE 8.0; Windows NT 6.1; WOW64; Trident/4.0; GTB7.2; SLCC2; .NET CLR 2.0.50727; .NET CLR 3.5.30729; .NET CLR 3.0.30729; Media Center PC 6.0; .NET4.0C)"
160.165.186.172 - - [04/Aug/2016:00:00:30 +0000] "GET /category/office HTTP/1.1" 200 101 "/category/electronics" "Mozilla/4.0 (compatible; MSIE 8.0; Windows NT 5.1; Trident/4.0; YTB720; GTB7.2; .NET CLR 1.1.4322; .NET CLR 2.0.50727; .NET CLR 3.0.4506.2152; .NET CLR 3.5.30729)"
224.150.219.97 - - [04/Aug/2016:00:00:35 +0000] "GET /category/jewelry HTTP/1.1" 200 74 "-" "Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)"

我已经在grokdebug中检查了过滤器,并且在那里一切正常,但是每次我将这些日志推送到logstash时,它都不会应用该过滤器,而是所有日志条目都带有“ _grokparsefailure”标签。

知道这里可能是什么问题吗? 我遵循了一些指南,但仍然存在此问题。

PS:我了解COMBINEDAPACHELOG,但我仍然想以自己的经验来解析它,并了解ELK堆栈击球手。

尝试在您的希腊人比赛messege更改为message

         change 'e' to 'a'
                 |
                 v
  match => ["message", "%{IP:client} - - \[%{HTTPDATE:event_date}\] %{QS:first} %{NUMBER:response} %{NUMBER:bytes} %{QS:destination} %{QS:browser}"]

暂无
暂无

声明:本站的技术帖子网页,遵循CC BY-SA 4.0协议,如果您需要转载,请注明本站网址或者原文地址。任何问题请咨询:yoyou2525@163.com.

 
粤ICP备18138465号  © 2020-2024 STACKOOM.COM