用于标记退回邮件的 Logstash grok 过滤器

Question

Summary: I've a few outbound smtp servers and centralized mail logs via rsyslog to a server on which i'm Using logstash, outputting to elasticsearch, searching with kibana.

I would like to tag as "BOUNCED" for Postfix mail log entries like:

2013-02-01T16:50:14+02:00 XXSMTPXX postfix/smtp[10879]: BC54A65BD4: to=<xxxx.yyyyyy@zzzz.com.t>, relay=none, delay=0.3, delays=0.01/0/0.29/0, dsn=5.4.4, status=bounced (Host or domain name not found. Name service error for name=gozdesigorta.com.t type=AAAA: Host not found)

the grok filter i've used in logstash.conf is like:

grok {
    patterns_dir => "/etc/logstash/patterns"
    tags    => "postfix/bounce"
    pattern => "%{POSTFIXBOUNCE}"
    add_tag => "BOUNCED"
    named_captures_only => true
}

patterns file i'm using is https://gist.github.com/4691822

I could not manage to tag those log lines as BOUNCED... What am i missing?

Answer 1

I changed the pattern as:

%{TIMESTAMP_ISO8601} %{HOST} %{SYSLOGPROG}: %{QUEUEID}: to=<%{EMAILADDRESS:to}>, relay=%{RELAY}, delay=%{POSREAL:delay}, delays=%{DELAYS}, dsn=%{DSN}, status=%{STATUS} %{GREEDYDATA:reason}

Now i can grok ;)

Answer 2

Can you paste the complete logstash.conf please? I get the following error:

Failed to execute action {:action=>LogStash::PipelineAction::Create/pipeline_id:main, :exception=>"LogStash::ConfigurationError", :message=>"Expected one of [ \\t\\r\\n], \"#\", \"=>\" at line 20, column 19 (byte 558) after filter {        \n  grok {\n      patterns_dir => \"/etc/logstash/conf.d/patterns\"\n      tags    => \"postfix/bounce\"\n      pattern => \"%{POSTFIXBOUNCE}\"\n      add_tag => \"%{TIMESTAMP_ISO8601} %{HOST} %{SYSLOGPROG}: %{QUEUEID}: to=<%{EMAILADDRESS:to}>, relay=%{RELAY}, delay=%{POSREAL:delay}, delays=%{DELAYS}, dsn=%{DSN}, status=%{STATUS} %{GREEDYDATA:reason}\"\n      named_captures_only => true\n  }\n\noutput {\n    elasticsearch ", :backtrace=>["/usr/share/logstash/logstash-core/lib/logstash/compiler.rb:32:in `compile_imperative'", "org/logstash/execution/AbstractPipelineExt.java:189:in `initialize'", "org/logstash/execution/JavaBasePipelineExt.java:72:in `initialize'", "/usr/share/logstash/logstash-core/lib/logstash/java_pipeline.rb:47:in `initialize'", "/usr/share/logstash/logstash-core/lib/logstash/pipeline_action/create.rb:52:in `execute'", "/usr/share/logstash/logstash-core/lib/logstash/agent.rb:383:in `block in converge_state'"]}