簡體   English   中英

在logstash中過濾json

[英]filter json in logstash

我有一個JSON文件,其中包含這樣的記錄

{"id":1,"first_name":"Frank","last_name":"Mills","date":"5/31/2014","email":"fmills0@feedburner.com","country":"France","city":"La Rochelle","latitude":"46.1667","longitude":"-1.15"

到目前為止,我一直在嘗試過濾Logstash中的字段,但未成功。 我嘗試了grok 調試器grokconstructor,但無法使其正常工作。 我最后的嘗試是

input {
    file{
        path => ["C:/logstash-1.4.2/mock_data.json"]
        type => "json"
        start_position => "beginning"
        sincedb_path => "/dev/null"
  }
}
filter {
  mutate {
    replace => [ "message", "%{message}" ]
  }
  json {
    source => "message"
    remove_field => "message"
  }
  mutate {
    convert => [ "latitude", "float" ]
    convert => [ "longitude","float" ]
  }
  mutate {
     rename => [ "latitude", "[location][lat]", "longitude", "[location][lon]" ]
  }
}

output {
  stdout {
    codec => rubydebug
  } 
  elasticsearch {
    host => "127.0.0.1"
    protocol => "http"
    index => "test35"
  }
} 

僅用於緯度和經度,但這不起作用。 特別是有關Json的logstash的任何教程。 任何幫助。 特定配置文件的輸出是

{
 "message" => "{\"id\":91,\"first_name\":\"Adam\",\"last_name\":\"Carr\",\"date\":\"11/14/2014\",\"email\":\"acarr2i@tinyurl.
com\",\"country\":\"Ghana\",\"city\":\"Mampong\",\"latitude\":\"7.06273\",\"longitude\":\"-1.4001\"},",
      "@version" => "1",
      "@timestamp" => "2015-05-04T19:05:08.409Z",
       "host" => "Toshiba",
       "path" => "C:/logstash-1.4.2/mock_data.json",
        "tags" => [
             [0] "_jsonparsefailure"
    ]
}

已針對Alcanzar更新

geoip過濾器用於將IP地址的經/緯度添加到數據中。

將所有部分放在一起將產生以下結果:

filter {
  grok {
        match => [ 'message', '(?<body>\"id\":.*\"longitude\":\"[^"]+\")' ]
        add_field => [ "json_body", "{%{body}}" ]
  }
  json {
        source => "json_body"
        remove_field => ["message","body","json_body" ]
  }
  mutate {
    convert => [ "latitude", "float" ]
    convert => [ "longitude","float" ]
  }
  mutate {
     rename => [ "latitude", "[location][lat]", 
       "longitude", "[location][lon]" ]
  }
}

它將生成一個如下所示的事件:

{
      "@version" => "1",
    "@timestamp" => "2015-05-04T19:48:52.051Z",
          "host" => "xxxxxxxx",
            "id" => 1,
    "first_name" => "Frank",
     "last_name" => "Mills",
          "date" => "5/31/2014",
         "email" => "fmills0@feedburner.com",
       "country" => "France",
          "city" => "La Rochelle",
      "location" => {
        "lat" => 46.1667,
        "lon" => -1.15
    }
}

這應該正是您想要的。

暫無
暫無

聲明:本站的技術帖子網頁,遵循CC BY-SA 4.0協議,如果您需要轉載,請注明本站網址或者原文地址。任何問題請咨詢:yoyou2525@163.com.

 
粵ICP備18138465號  © 2020-2024 STACKOOM.COM