JSON数组解析问题与logstash

Question

We want to implement service request trace using http plugin of logstash in JSON Array format.

We are getting the following error when trying to parse the JSON array:

error:

:message=>"gsub mutation is only applicable for Strings, skipping", :field=>"message", :value=>nil, :level=>:debug, :file=>"logstash/filters/mutate.rb", :line=>"322", :method=>"gsub"}
:message=>"Exception in filterworker", "exception"=>#<LogStash::ConfigurationError: Only String and Array types are splittable. field:message is of type = NilClass>

My json array is :

{
    "data": [
        {
            "appName": "DemoApp",
            "appVersion": "1.1",
            "deviceId": "1234567",
            "deviceName": "moto e",
            "deviceOSVersion": "5.1",
            "packageName": "com.DemoApp",
            "message": "testing null pointer exception",
            "errorLog": "null pointer exception"
        },
        {
            "appName": "DemoApp",
            "appVersion": "1.1",
            "deviceId": "1234567",
            "deviceName": "moto e",
            "deviceOSVersion": "5.1",
            "packageName": "com.DemoApp",
            "message": "testing illegal state exception",
            "errorLog": "illegal state exception"
        }
    ]
}

my logstash config is :

    input {
            http {
            codec => "plain"
            }
    }
    filter{
            json {
                  source => "message"
                 }
            mutate { gsub => [ "message", "},", "shr" ] }
            split {
                  terminator => "shr"
                  field => "data"
           }
    }
    }


output {
 stdout { codec => "json" }
    gelf{
        host => localhost
        facility => "%{type}"
        level =>["%{SeverityLevel}", "INFO"]
        codec => "json"
    }
       file{
        path => "/chroot/result.log"
}
}

Any help would be appreciated.

Answer 1

Logstash has a default metadata field named message . So your json message field is overlapping that. Consider changing json field name message to another.

The other option maybe using target setting and referencing the target field like:

json { source => "message" target => "data"}
mutate { gsub => [ "[data][message]", "\}\,\r\n\r\n\{", "\}shr\{" ] }

I hope this helps.