json输入文件的logstash问题

Question

我在文件中有以下json-

{
"foo":"bar",
"spam" : "eggs"
},
{
"css":"ddq",
"eeqw": "fewq"
}

和以下conf文件-

input { 
file
{ 
   path => "/opt/logstash-1.4.2/bin/sam.json"
   type => "json"
   codec => json_lines
   start_position =>"beginning"
 }
}
output { stdout {  codec => json  } }

但是当我跑步时

./logstash -f sample.conf

我在标准输出中没有任何输出。

但是当我不给json作为编解码器并给类型=>“ core2”时，它似乎可以工作。 任何人都知道我如何修复它以适用于json类型。

另一个问题是，当它给出stdout-时，它给了我以下输出-

{"message":"{","@version":"1","@timestamp":"2015-07-15T02:02:02.653Z","type":"core2","host":"sjagannath","path":"/opt/logstash-1.4.2/bin/sam.json"}{"message":"\"foo\":\"bar\", ","@version":"1","@timestamp":"2015-07-15T02:02:02.654Z","type":"core2","host":"sjagannath","path":"/opt/logstash-1.4.2/bin/sam.json"}{"message":"\"spam\" : \"eggs\" ","@version":"1","@timestamp":"2015-07-15T02:02:02.655Z","type":"core2","host":"sjagannath","path":"/opt/logstash-1.4.2/bin/sam.json"}{"message":"},","@version":"1","@timestamp":"2015-07-15T02:02:02.655Z","type":"core2","host":"sjagannath","path":"/opt/logstash-1.4.2/bin/sam.json"}{"message":"{ ","@version":"1","@timestamp":"2015-07-15T02:02:02.655Z","type":"core2","host":"sjagannath","path":"/opt/logstash-1.4.2/bin/sam.json"}{"message":"\"css\":\"ddq\", ","@version":"1","@timestamp":"2015-07-15T02:02:02.656Z","type":"core2","host":"sjagannath","path":"/opt/logstash-1.4.2/bin/sam.json"}{"message":"\"eeqw\": \"fewq\"","@version":"1","@timestamp":"2015-07-15T02:02:02.656Z","type":"core2","host":"sjagannath","path":"/opt/logstash-1.4.2/bin/sam.json"}{"message":"}","@version":"1","@timestamp":"2015-07-15T02:02:02.656Z","type":"core2","host":"sjagannath","path":"/opt/logstash-1.4.2/bin/sam.json"}{"message":"","@version":"1","@timestamp":"2015-07-15T02:02:02.656Z","type":"core2","host":"sjagannath","path":"/opt/logstash-1.4.2/bin/sam.json"}

我想知道如何用输入文件中的键值对正确解析它

Answer 1

我找到了这个，并对其进行了编辑以适合您的目的。 以下配置应完全满足您的要求：

input {   
file     {
    codec => multiline
    {
        pattern => "^\}"
        negate => true
        what => previous               
    }
    path => ["/absoute_path/json.json"]
    start_position => "beginning"
    sincedb_path => "/dev/null"
}
}

filter {
mutate    {
    replace => [ "message", "%{message}}" ]
    gsub => [ "message","\n",""]
    gsub => [ "message","},",""]
}
if [message] =~ /^{.*}$/     {
    json { source => message }
}
}

我尝试了给定的json，它导致了两个事件。 首先使用foo = bar和spam = eggs 。 其次是css = ddq eeqw = fewq和eeqw = fewq 。

Answer 2

据我了解，如果您想使用json_lines编解码器，则希望将完整的子文档放在一行上：

{"foo":"bar","spam" : "eggs"}
{"css":"ddq","eeqw": "fewq"}

在您的情况下，您的结构存在问题，因为子对象之间也有一个“，”。 不是处理它的最简单方法。 因此，如有可能，将源更改为我的示例。 如果不可能，那么多行方法可能会为您提供帮助。 检查以供参考：将json输入到logstash-配置问题？