将 geoIP 数据添加到来自 Elasticsearch 索引的旧数据

Question

I recently added a GeoIP processor to my ingestion pipeline in Elasticsearch. this works well and adds new fields to the newly ingested documents. I wanted to add the GeoIP fields to older data by doing an _update_by_query on an index, however, it seems that it doesn't accept "processors" as a parameter.

What I want to do is something like this:

POST my_index*/_update_by_query
{
 "refresh": true,
 "processors": [
   {
     "geoip" : {
        "field": "doc['client_ip']",
        "target_field" : "geo",
        "database_file" : "GeoLite2-City.mmdb",
        "properties":["continent_name", "country_iso_code", "country_name", "city_name", "timezone", "location"]
    }
   }
 ],
 "script": {
  "day_of_week": {
    "type": "long",
    "script": "emit(doc['@timestamp'].value.withZoneSameInstant(ZoneId.of(doc['geo.timezone'])).getDayOfWeek().getValue())"
  },
  "hour_of_day": {
    "type": "long",
    "script": "emit(doc['@timestamp'].value.withZoneSameInstant(ZoneId.of(doc['geo.timezone'])).getHour())"
  },
  "office_hours": {
    "script": "if (doc['day_of_week'].value< 6 && doc['day_of_week'].value > 0) {if (doc['hour_of_day'].value> 7 && doc['hour_of_day'].value<19) {return 1;} else {return -1;} } else {return -1;}"
  }
 }
}

I receive the following error:

{
  "error" : {
    "root_cause" : [
      {
        "type" : "parse_exception",
        "reason" : "Expected one of [source] or [id] fields, but found none"
      }
    ],
    "type" : "parse_exception",
    "reason" : "Expected one of [source] or [id] fields, but found none"
  },
  "status" : 400
}

Answer 1

Since you have the ingestion pipeline ready, you simply need to reference it in your call to the _update_by_query endpoint, like this:

POST my_index*/_update_by_query?pipeline=my-pipeline
                                    ^
                                    |
                                 add this