json輸入文件的logstash問題

Question

我在文件中有以下json-

{
"foo":"bar",
"spam" : "eggs"
},
{
"css":"ddq",
"eeqw": "fewq"
}

和以下conf文件-

input { 
file
{ 
   path => "/opt/logstash-1.4.2/bin/sam.json"
   type => "json"
   codec => json_lines
   start_position =>"beginning"
 }
}
output { stdout {  codec => json  } }

但是當我跑步時

./logstash -f sample.conf

我在標准輸出中沒有任何輸出。

但是當我不給json作為編解碼器並給類型=>“ core2”時，它似乎可以工作。 任何人都知道我如何修復它以適用於json類型。

另一個問題是，當它給出stdout-時，它給了我以下輸出-

{"message":"{","@version":"1","@timestamp":"2015-07-15T02:02:02.653Z","type":"core2","host":"sjagannath","path":"/opt/logstash-1.4.2/bin/sam.json"}{"message":"\"foo\":\"bar\", ","@version":"1","@timestamp":"2015-07-15T02:02:02.654Z","type":"core2","host":"sjagannath","path":"/opt/logstash-1.4.2/bin/sam.json"}{"message":"\"spam\" : \"eggs\" ","@version":"1","@timestamp":"2015-07-15T02:02:02.655Z","type":"core2","host":"sjagannath","path":"/opt/logstash-1.4.2/bin/sam.json"}{"message":"},","@version":"1","@timestamp":"2015-07-15T02:02:02.655Z","type":"core2","host":"sjagannath","path":"/opt/logstash-1.4.2/bin/sam.json"}{"message":"{ ","@version":"1","@timestamp":"2015-07-15T02:02:02.655Z","type":"core2","host":"sjagannath","path":"/opt/logstash-1.4.2/bin/sam.json"}{"message":"\"css\":\"ddq\", ","@version":"1","@timestamp":"2015-07-15T02:02:02.656Z","type":"core2","host":"sjagannath","path":"/opt/logstash-1.4.2/bin/sam.json"}{"message":"\"eeqw\": \"fewq\"","@version":"1","@timestamp":"2015-07-15T02:02:02.656Z","type":"core2","host":"sjagannath","path":"/opt/logstash-1.4.2/bin/sam.json"}{"message":"}","@version":"1","@timestamp":"2015-07-15T02:02:02.656Z","type":"core2","host":"sjagannath","path":"/opt/logstash-1.4.2/bin/sam.json"}{"message":"","@version":"1","@timestamp":"2015-07-15T02:02:02.656Z","type":"core2","host":"sjagannath","path":"/opt/logstash-1.4.2/bin/sam.json"}

我想知道如何用輸入文件中的鍵值對正確解析它

Answer 1

我找到了這個，並對其進行了編輯以適合您的目的。 以下配置應完全滿足您的要求：

input {   
file     {
    codec => multiline
    {
        pattern => "^\}"
        negate => true
        what => previous               
    }
    path => ["/absoute_path/json.json"]
    start_position => "beginning"
    sincedb_path => "/dev/null"
}
}

filter {
mutate    {
    replace => [ "message", "%{message}}" ]
    gsub => [ "message","\n",""]
    gsub => [ "message","},",""]
}
if [message] =~ /^{.*}$/     {
    json { source => message }
}
}

我嘗試了給定的json，它導致了兩個事件。 首先使用foo = bar和spam = eggs 。 其次是css = ddq eeqw = fewq和eeqw = fewq 。

Answer 2

據我了解，如果您想使用json_lines編解碼器，則希望將完整的子文檔放在一行上：

{"foo":"bar","spam" : "eggs"}
{"css":"ddq","eeqw": "fewq"}

在您的情況下，您的結構存在問題，因為子對象之間也有一個“，”。 不是處理它的最簡單方法。 因此，如有可能，將源更改為我的示例。 如果不可能，那么多行方法可能會為您提供幫助。 檢查以供參考：將json輸入到logstash-配置問題？