Logstash無法在JSON中正確轉換

Question

以下是我的json日志文件

[
    {
        "error_message": " Failed to get line from input file (end of file?).", 
        "type": "ERROR", 
        "line_no": "2625", 
        "file": "GTFplainText.c", 
        "time": "17:40:02", 
        "date": "01/07/16", 
        "error_code": "GTF-00014"
    }, 
    {
        "error_message": " Bad GTF plain text file header or footer line. ", 
        "type": "ERROR", 
        "line_no": "2669", 
        "file": "GTFplainText.c", 
        "time": "17:40:02", 
        "date": "01/07/16", 
        "error_code": "GTF-00004"
    }, 
    {
        "error_message": " '???' ", 
        "type": "ERROR", 
        "line_no": "2670", 
        "file": "GTFplainText.c", 
        "time": "17:40:02", 
        "date": "01/07/16", 
        "error_code": "GTF-00005"
    }, 
    {
        "error_message": " Failed to find 'event source'/'product detail' records for event source '3025188506' host event type 1 valid", 
        "type": "ERROR", 
        "line_no": "0671", 
        "file": "RGUIDE.cc", 
        "time": "15:43:48", 
        "date": "06/07/16", 
        "error_code": "RGUIDE-00033"
    }
]

根據我的理解，由於日志已經在json中，因此我們在logstash配置中不需要filter部分。 以下是我的logstash配置

input {
  file{
    path => "/home/ishan/sf_shared/log_json.json"
    start_position => "beginning"
    codec => "json"
  }
}

並且輸出配置是

output {
  elasticsearch {
    hosts => ["localhost:9200"]
    sniffing => true
    manage_template => false
    index => "%{[@metadata][beat]}-%{+YYYY.MM.dd}"
    document_type => "%{[@metadata][type]}"
  }
  stdout { codec => rubydebug }
}

但是似乎數據沒有進入ES，因為查詢索引時我看不到數據。 我想念什么？

Answer 1

我認為問題是json編解碼器期望一行上有完整的json消息，而不能在多行上使用一條消息。

可能的解決方法是使用多行編碼並使用json過濾器 。
多行編解碼器的配置為：

multiline {
  pattern => "]"
  negate => "true"
  what => "next"
}

所有不以]開頭的行將與下一行重新組合，因此您將擁有一個完整的json文檔以提供給json過濾器。