[英]filter json in logstash
我有一個JSON文件,其中包含這樣的記錄
{"id":1,"first_name":"Frank","last_name":"Mills","date":"5/31/2014","email":"fmills0@feedburner.com","country":"France","city":"La Rochelle","latitude":"46.1667","longitude":"-1.15"
到目前為止,我一直在嘗試過濾Logstash中的字段,但未成功。 我嘗試了grok 調試器和grokconstructor,但無法使其正常工作。 我最后的嘗試是
input {
file{
path => ["C:/logstash-1.4.2/mock_data.json"]
type => "json"
start_position => "beginning"
sincedb_path => "/dev/null"
}
}
filter {
mutate {
replace => [ "message", "%{message}" ]
}
json {
source => "message"
remove_field => "message"
}
mutate {
convert => [ "latitude", "float" ]
convert => [ "longitude","float" ]
}
mutate {
rename => [ "latitude", "[location][lat]", "longitude", "[location][lon]" ]
}
}
output {
stdout {
codec => rubydebug
}
elasticsearch {
host => "127.0.0.1"
protocol => "http"
index => "test35"
}
}
僅用於緯度和經度,但這不起作用。 特別是有關Json的logstash的任何教程。 任何幫助。 特定配置文件的輸出是
{
"message" => "{\"id\":91,\"first_name\":\"Adam\",\"last_name\":\"Carr\",\"date\":\"11/14/2014\",\"email\":\"acarr2i@tinyurl.
com\",\"country\":\"Ghana\",\"city\":\"Mampong\",\"latitude\":\"7.06273\",\"longitude\":\"-1.4001\"},",
"@version" => "1",
"@timestamp" => "2015-05-04T19:05:08.409Z",
"host" => "Toshiba",
"path" => "C:/logstash-1.4.2/mock_data.json",
"tags" => [
[0] "_jsonparsefailure"
]
}
已針對Alcanzar更新
geoip
過濾器用於將IP地址的經/緯度添加到數據中。
將所有部分放在一起將產生以下結果:
filter {
grok {
match => [ 'message', '(?<body>\"id\":.*\"longitude\":\"[^"]+\")' ]
add_field => [ "json_body", "{%{body}}" ]
}
json {
source => "json_body"
remove_field => ["message","body","json_body" ]
}
mutate {
convert => [ "latitude", "float" ]
convert => [ "longitude","float" ]
}
mutate {
rename => [ "latitude", "[location][lat]",
"longitude", "[location][lon]" ]
}
}
它將生成一個如下所示的事件:
{
"@version" => "1",
"@timestamp" => "2015-05-04T19:48:52.051Z",
"host" => "xxxxxxxx",
"id" => 1,
"first_name" => "Frank",
"last_name" => "Mills",
"date" => "5/31/2014",
"email" => "fmills0@feedburner.com",
"country" => "France",
"city" => "La Rochelle",
"location" => {
"lat" => 46.1667,
"lon" => -1.15
}
}
這應該正是您想要的。
聲明:本站的技術帖子網頁,遵循CC BY-SA 4.0協議,如果您需要轉載,請注明本站網址或者原文地址。任何問題請咨詢:yoyou2525@163.com.