在Logstash中分解json数组字符串

Question

I have a RabbitMQ sending 3 fields in JSON format and it is being consumed by Logstash rabbitmq input plugin.

One of the fields is a JSON array of strings as follows:

"content": [
  "1111",
  "2222222",
  "Test 06",
  "3",
  "3232",
  "SomeValue1"
]

1. How could I make each entry of that string into a field so that I can quickly discover and visualize in Kibana from the available fields? Right now I see "content" with that full string.

2. The JSON array string size changes depending on another field, eventID. Would it be possible to dynamically map the values in that string to specific names depending on the eventID? Such as:

    "eventID": 1, "content": [ "name1": "1111", "name2": "2222222", "name3": "Test 06", "name4": "3", "name5": "3232", "name6": "SomeValue1" ] "eventID": 2, "content": [ "othername1": "3434", "othername2": "Test 10", "othername3": "876", "othername4": "Some String7" ] 

I would like to have the name* and othername* in the available fields. Any help would be appreciated.

Answer 1

First, I'm going to assume for a moment that your input is properly parsed as an array already. For testing purposes, that means that this:

echo '{"eventID":1,"content":["a","b","c","d"]}' | bin/logstash -f test.conf 

where test.conf is:

input {
    stdin { codec => json }
}
output {
    stdout { codec => rubydebug }
}

will output this:

{
       "eventID" => 1,
    "@timestamp" => 2017-08-03T19:39:13.054Z,
      "@version" => "1",
          "host" => "xxxxxx.local",
       "content" => [
        [0] "a",
        [1] "b",
        [2] "c",
        [3] "d"
    ]
}

If that's the case, then you'll need to do something like this:

filter {
    if [eventID] == 1 {
        mutate {
            add_field => {
                "eventName" => "type1 event"
                "one0" => "%{[content][0]}"
                "one1" => "%{[content][1]}"
                "one2" => "%{[content][2]}"
                "one3" => "%{[content][3]}"
            }
            remove_field => [ "content" ]
        }
    } else if [eventID] == 2 {
        mutate {
            add_field => {
                "eventName" => "type2 event"
                "two0" => "%{[content][0]}"
                "two1" => "%{[content][1]}"
                "two2" => "%{[content][2]}"
                "two3" => "%{[content][3]}"
            }
            remove_field => [ "content" ]
        }
    }
}

which will generate an event like this:

{
       "eventID" => 1,
    "@timestamp" => 2017-08-03T19:51:02.946Z,
      "@version" => "1",
          "host" => "xxxxxxx.local",
          "one2" => "c",
     "eventName" => "type1 event",
          "one3" => "d",
          "one0" => "a",
          "one1" => "b"
}