在 logstash 中过滤日志并保存到 Elastic Search

Question

I am dumping all my logs into Elastic Search. It looks like.

Now I want:

1. Only limited fields should be pushed to ES.
2. Parse message part and add another field log.level: and save the value INFO/DEBUG/ERROR from the message field.

Can anyone help me to do that. I am new to Elastic.

My logstash.conf is

 input {
      beats {
        port => 5044
       }
    }
 
output {
        if [agent][hostname]=="TEST-PC"{
                elasticsearch {
                        hosts => ["localhost:9200"]
                        manage_template => false
                        index => "INDEXNAME"
                        user => "elastic"
                        password => "password"
              }
          }     
      }

Answer 1

Add this filter section to your logstash config file.

filter {
      grok {
        match => { "message" => "%{TIMESTAMP_ISO8601}\s\[%{NOTSPACE:Meta}\]\s%{WORD:Logtype}\s%{GREEDYDATA:Log_Message}" }
      }
      if "INFO" in [Logtype] {
          drop{}
      }
  }