Getting _jsonparsefailure with valid json in logstash

Question

I currently and attempting to send a json through logstash to elasticsearch. However, I am getting an error that there is a jsonparsefailure even though I have validated my json as being in the proper format. What could be my issue? Below is my config file and my json

input{
 file{
        path => "/Users/CFP668/Downloads/gs-accessing-data-mysql-master/complete/example.json"
        codec => json
        start_position => "beginning"
 }
}

filter{
    json{
        source => "message"
    }
}

output{
    elasticsearch{
        hosts => "localhost:9200"
        index => "test123" 
    }
    stdout { codec => rubydebug }
}

"{\"request_id\":1,\"requestDetail_id\":0,\"enrichedContent\":\"enrich cont example\",\"statusCode\":\"P\",\"reasonDesc\":\"This is a test\",\"createdBy\":\"Sarah\",\"createdDate\":\"Sep 24, 2010\"}"

Answer 1

Your json isn't really valid for Logstash, you have a backslash before the double quotes on your keys and your json object is also between double quotes.

Your json lines on the source file should be something like this:

{"request_id":1,"requestDetail_id":0,"enrichedContent":"enrich cont example","statusCode":"P","reasonDesc":"This is a test","createdBy":"Sarah","createdDate":"Sep 24, 2010"}

The way to work with your current json line, without changing how it is generated, is to use a gsub on you message to change "{ to { , }" to } and to remove the backslashes \\

The following filter pipeline does that:

filter {
    mutate {
        gsub => ["message","^\"{","{"]
        gsub => ["message","}\"$","}"]
        gsub => ["message","[\\]",""]
    }
    json {
        source => "message"
    }
}

And you can remove the codec => "json" from your input block.