Error in grok filter which starting logstash

Question

I have the following logstash conf file

input {
  tcp {
    port => 12345
    codec => json
  }
}

filter {
  grok {
    break_on_match => true
    match => [
        "message", "%{TIMESTAMP_ISO8601:timestamp} (verbose|info|debug) (hostd|vpxa)",
    ]
    mutate {
      add_tag => "esxi_verbose"
    }
  }
}

if "esxi_verbose" in [tags] {
  drop{}
}

output {
      stdout { codec => rubydebug }
      elasticsearch { 
        hosts => ["localhost:9200"] 
        index => "logstash-%{+YYYY.MM.dd}"
      }
}

I am trying to drop any verbose, debug, info messages. When I start logstash I get the error

[2019-03-03T16:53:11,731][ERROR][logstash.agent] Failed to execute action {:action=>LogStash::PipelineAction::Create/pipeline_id:main, :exception=>"LogStash::ConfigurationError", :message=>"Expected one of #, \", ', -, [, { at line 13, column 5 (byte 211) after filter {\n  grok {\n    break_on_match => true\n    match => [\n        \"message\", \"%{TIMESTAMP_ISO8601:timestamp} (verbose|info|debug) (hostd|vpxa)\",\n    "

Can someone help me what I am doing wrong.

Answer 1

you have 3 issues in the config:

1. there's a comma at the end of the grok message line which is redundant
2. the mutate is inside the grok filter, but it should come after it
3. the 'if' statement should be inside the 'filter' section.

This is the updated and working config:

input {
  tcp {
    port => 12345
    codec => json
  }
}

filter {
  grok {
    break_on_match => true
    match => [
        "message", "%{TIMESTAMP_ISO8601:timestamp} (verbose|info|debug) (hostd|vpxa)"
    ]
  }

  mutate {
    add_tag => "esxi_verbose"
  }

  if "esxi_verbose" in [tags] {
    drop{}
  }

}

output {
      stdout { codec => rubydebug }
      elasticsearch {
        hosts => ["localhost:9200"]
        index => "logstash-%{+YYYY.MM.dd}"
      }
}