Parse nested json in logstash

Question

I am parsing this log in logstash. This log contains nested json.

[root@localhost conf.d]# cat dummy.txt
{"dwo":"0","rs":{"$date":"2016-02-11T00:52:50.348Z"},"first":[{"timestamp":"2016-02-11T00:52:51Z","a":"abc","b":"243253456","c":"fdsfg444323fef"},{"timestamp":"2016-02-11T00:52:51Z","a":"xyz","b":"543q653q654","c":"fewdfsdf","d":"Thu Feb 11 2016 11:52:47 GMT 1100 (AUS Eastern Daylight Time)","e":"4235345"}],"bwv":"40","cp_tz":{"$date":"2016-02-11T11:52:50.000Z"}}

Here is my logstash configuration

[root@localhost conf.d]# cat hadoop.conf
input {
    stdin {}
}

filter {
    json {
            source => message
    }
}

output {
    stdout { codec => rubydebug }
}

and this is how I am running logstash

[root@localhost conf.d]# /opt/logstash/bin/logstash -f /etc/logstash/conf.d/hadoop.conf
Logstash startup completed
{"dwo":"0","rs":{"$date":"2016-02-11T00:52:50.348Z"},"first":[{"timestamp":"2016-02-11T00:52:51Z","a":"abc","b":"243253456","c":"fdsfg444323fef"},{"timestamp":"2016-02-11T00:52:51Z","a":"xyz","b":"543q653q654","c":"fewdfsdf","d":"Thu Feb 11 2016 11:52:47 GMT 1100 (AUS Eastern Daylight Time)","e":"4235345"}],"bwv":"40","cp_tz":{"$date":"2016-02-11T11:52:50.000Z"}}
{
   "message" => "{\"dwo\":\"0\",\"rs\":{\"$date\":\"2016-02-11T00:52:50.348Z\"},\"first\":[{\"timestamp\":\"2016-02-11T00:52:51Z\",\"a\":\"abc\",\"b\":\"243253456\",\"c\":\"fdsfg444323fef\"},{\"timestamp\":\"2016-02-11T00:52:51Z\",\"a\":\"xyz\",\"b\":\"543q653q654\",\"c\":\"fewdfsdf\",\"d\":\"Thu Feb 11 2016 11:52:47 GMT 1100 (AUS Eastern Daylight Time)\",\"e\":\"4235345\"}],\"bwv\":\"40\",\"cp_tz\":{\"$date\":\"2016-02-11T11:52:50.000Z\"}}",
  "@version" => "1",
"@timestamp" => "2016-04-04T03:04:17.722Z",
      "host" => "localhost",
       "dwo" => "0",
        "rs" => {
    "$date" => "2016-02-11T00:52:50.348Z"
},
     "first" => [
    [0] {
        "timestamp" => "2016-02-11T00:52:51Z",
                "a" => "abc",
                "b" => "243253456",
                "c" => "fdsfg444323fef"
    },
    [1] {
        "timestamp" => "2016-02-11T00:52:51Z",
                "a" => "xyz",
                "b" => "543q653q654",
                "c" => "fewdfsdf",
                "d" => "Thu Feb 11 2016 11:52:47 GMT 1100 (AUS Eastern Daylight Time)",
                "e" => "4235345"
    }
],
       "bwv" => "40",
     "cp_tz" => {
    "$date" => "2016-02-11T11:52:50.000Z"
}
}

If you look at the below image, you will find that the component inside the "first" json is not indexed hence not searchable. That is what my query is. I am not able to parse nested json.

!kibana]https://cdn.discourse.org/elastic/uploads/default/optimized/2X/e/e71d3a6734a6b49e3722b0b140d9ae77711ab73a_1_690x381.png

Answer 1

Kibana不支持（也不会）支持对象数组。