简体   繁体   English

在Logstash过滤器中解释Keen.io JSON文件中的位置

[英]Interpret locations from Keen.io JSON file in logstash filter

I'm trying to parse a JSON file from Keen.io with logstash into elasticsearch. 我正在尝试将带有logstash的Keen.io的JSON文件解析为elasticsearch。 The location and timestamp are stored in parameters like this: 位置和时间戳存储在这样的参数中:

{
  "result":
  [
    {
      "keen":
      {
        "timestamp": "2014-12-02T12:23:51.000Z",
        "created_at": "2014-12-01T23:25:31.396Z",
        "id": "XXXX",
        "location":
        {
          "coordinates": [-95.8, 36.1]
        }
      }
    }
  ]
}

My filter currently looks like this: 我的过滤器当前如下所示:

input {
  file {
    path => ["test.json"]
    start_position => beginning
    type => json
  }
}

filter {
  json {
    source => message
    remove_field => message
  }
}

output {
  stdout { codec => rubydebug }
}

How can I parse the "timestamp" and "location" fields so they are used for the @timestamp and @geoip.coordinates in Elasticsearch? 如何解析“ timestamp”和“ location”字段,以便将它们用于Elasticsearch中的@timestamp和@ geoip.coordinates?

Update: I've tried variations of this with no luck. 更新:我已经尝试过这种变化,没有运气。 The documentation is very basic - am I misunderstanding how to reference the JSON fields? 该文档非常基础-我是否误解了如何引用JSON字段? Is there a way of adding debug output to help? 有没有添加调试输出以帮助的方法? I tried How to debug the logstash file plugin and Print a string to stdout using Logstash 1.4? 我尝试过如何 使用Logstash 1.4 调试logstash文件插件 并向stdout打印字符串? but neither works. 但都行不通。

filter {
  json {
    source => message
    remove_field => message
  }
  if ("[result][0][keen][created_at]") {
    date {
      add_field => [ "[timestamp]", "[result][0][keen][created_at]" ]
      remove_field => "[result][0][keen][created_at]"
    }
  }

Update 2: 更新2:

Date is working now, still need to get location working. 日期正在工作,仍然需要使位置工作。

filter {
  json {
    source => message
    remove_field => message
    add_tag => ["valid_json"]
  }
  if ("valid_json") {
    if ("[result][0][keen][created_at]") {
      date {
        match => [ "[result][0][keen][created_at]", "ISO8601" ]
      }
    }
  }
}

Keen.io's "created_at" field is stored in ISO 8601 format and so can easily be parsed by the date filter. Keen.io的“ created_at”字段以ISO 8601格式存储,因此可以通过日期过滤器轻松解析。 Lat/long co-ordinates can be set by copying Keen.io's existing co-ordinates into logstash's geoip.coordinates array. 可以通过将Keen.io的现有坐标复制到logstash的geoip.coordinates数组中来设置纬度/经度坐标。

input {
  file {
    path => ["data.json"]
    start_position => beginning
    type => json
  }
}

filter {
  json {
    source => message
    remove_field => message
    add_tag => ["valid_json"]
  }
  if ("valid_json") {
    if ("[result][0][keen][created_at]") {
      date {
        # Set @timestamp to Keen.io's "created_at" field
        match => [ "[result][0][keen][created_at]", "ISO8601" ]
      }
    }
    if ("[result][0][keen][location][coordinates]") {
      mutate {
        # Copy existing co-orndiates into geoip.coordinates array
        add_field => [ "[geoip][coordinates]", "%{[result][0][keen][location][coordinates][0]}" ]
        add_field => [ "[geoip][coordinates]", "%{[result][0][keen][location][coordinates][1]}" ]
        remove_field => "[result][0][keen][location][coordinates]"
      }
    }
  }
}

output {
  stdout { codec => rubydebug }
}

声明:本站的技术帖子网页,遵循CC BY-SA 4.0协议,如果您需要转载,请注明本站网址或者原文地址。任何问题请咨询:yoyou2525@163.com.

 
粤ICP备18138465号  © 2020-2024 STACKOOM.COM