logstash json filter not parsing fields getting _jsonparsefailure

Question

Hi I am trying to parse a json file. I have tried troubleshooting with suggestions from stackoverflow (links at bottom)but none have worked for me. I am hoping someone has some insight on probably a silly mistake I am making.

I have tried using only the json codec, only the json filter, as well as both. For some reason I am still getting this _jsonparsefailure. What can I do to get this to work?

Thanks in advance!

My json file:

{
  "log": {
    "version": "1.2",
    "creator": {
      "name": "WebInspector",
      "version": "537.36"
    },
    "pages": [
      {
        "startedDateTime": "2015-10-13T20:28:46.081Z",
        "id": "page_1",
        "title": "https://demo.com",
        "pageTimings": {
          "onContentLoad": 377.8560000064317,
          "onLoad": 377.66200001351535
        }
      },
      {
        "startedDateTime": "2015-10-13T20:29:01.734Z",
        "id": "page_2",
        "title": "https://demo.com",
        "pageTimings": {
          "onContentLoad": 1444.0670000039972,
          "onLoad": 2279.20100002666
        }
      },
      {
        "startedDateTime": "2015-10-13T20:29:04.014Z",
        "id": "page_3",
        "title": "https://demo.com",
        "pageTimings": {
          "onContentLoad": 1802.0240000041667,
          "onLoad": 2242.4060000048485
        }
      },
      {
        "startedDateTime": "2015-10-13T20:29:09.224Z",
        "id": "page_4",
        "title": "https://demo.com",
        "pageTimings": {
          "onContentLoad": 274.82699998654425,
          "onLoad": 1453.034000005573
        }
      }
    ]
  }
}

My logstash conf:

input {
    file {
       type => "json"
       path => "/Users/anonymous/Documents/demo.json"
       start_position => beginning
    }
}

filter{
    json{
        source => "message"
   }
}


output { 
  elasticsearch { host => localhost protocol => "http" port => "9200" } 
  stdout { codec => rubydebug } 
}

Output I am getting from logstash hopefully with clues:

Trouble parsing json {:source=>"message", :raw=>"        \"startedDateTime\": \"2015-10-19T18:05:37.887Z\",", :exception=>#<TypeError: can't convert String into Hash>, :level=>:warn}
{
       "message" => "      {",
      "@version" => "1",
    "@timestamp" => "2015-10-26T20:05:53.096Z",
          "host" => "15mbp-09796.local",
          "path" => "/Users/anonymous/Documents/demo.json",
          "type" => "json",
          "tags" => [
        [0] "_jsonparsefailure"
    ]
}

Decompose Logstash json message into fields

How to use logstash's json filter?

Answer 1

I test my JSON here JSONLint . Perhaps this will solve your problem. The error I am getting is that it is expecting string.

It seems that you have an unnecessary comma(',') at the end. Either remove it or add another JSON variable after that.