简体   繁体   中英

Logstash cannot extract json key

I need help regarding logstash filter to extract json key/value to new_field. The following is my logstash conf.

input {
    tcp {
        port => 5044
    }
}

filter {
    json {
        source => "message"
        add_field => {
            "data" => "%{[message][data]}"
        }
    }
}

output {
        stdout { codec => rubydebug }
}

I have tried with mutate:

filter {
    json {
        source => "message"
    }
    mutate {
        add_field => {
            "data" => "%{[message][data]}"
        }
    }
}

I have tried with . instead of []:

filter {
    json {
        source => "message"
    }
    mutate {
        add_field => {
            "data" => "%{message.data}"
        }
    }
}

I have tried with index number:

filter {
    json {
        source => "message"
    }
    mutate {
        add_field => {
            "data" => "%{[message][0]}"
        }
    }
}

All with no luck. :(

The following json is sent to port 5044:

{"data": "blablabla"}

The problem is the new field not able to extract value from the key of the json.
"data" => "%{[message][data]}"

The following is my stdout:

{
           "@version" => "1",
               "host" => "localhost",
               "type" => "logstash",
               "data" => "%{[message][data]}",
               "path" => "/path/from/my/app",
         "@timestamp" => 2019-01-11T20:39:10.845Z,
            "message" => "{\"data\": \"blablabla\"}"
}

However if I use "data" => "%{[message]}" instead:

filter {
    json {
        source => "message"
        add_field => {
            "data" => "%{[message]}"
        }
    }
}

I will get the whole json from stdout.

{
           "@version" => "1",
               "host" => "localhost",
               "type" => "logstash",
               "data" => "{\"data\": \"blablabla\"}",
               "path" => "/path/from/my/app",
         "@timestamp" => 2019-01-11T20:39:10.845Z,
            "message" => "{\"data\": \"blablabla\"}"
}

Can anyone please tell me what I did wrong.
Thank you in advance.
I use docker-elk stack, ELK_VERSION=6.5.4

add_field is used to add custom logic when filter succeeds, many filters have this option. If you want to parse json into a field, you should use target :

filter {
  json {
    source => "message"
    target => "data"  // parse into data field
  }
}

The technical post webpages of this site follow the CC BY-SA 4.0 protocol. If you need to reprint, please indicate the site URL or the original address.Any question please contact:yoyou2525@163.com.

 
粤ICP备18138465号  © 2020-2024 STACKOOM.COM