Elasticsearch如何检查批量索引请求的状态？

Question

I am bulk indexing into Elasticsearch docs containing country shapes ( files here ), based on the cshapes dataset.

The geoshapes have a lot of points in "geometry":{"type":"MultiPolygon" , and the bulk request takes a long time to complete (and sometimes does not complete, which is a separate and already reported problem).

Since the client times out (I use the official ES node.js), I would like to have a way to check what the status of the bulk request is, without having to use enormous timeout values.

What I would like is to have a status such as active/running, completed or aborted. I guess that just by querying the single doc in the batch would not tell me whether the request has been aborted.

Is this possible?

Answer 1

Elasticsearch doesn't provide a way to check the status of an ongoing Bulk request- documentation reference here .

First, check that your request succeeds with a smaller input, so you know there is no problem with the way you are making the request. Second, try dividing the data into smaller chunks and calling the Bulk API on them in parallel.

You can also try with a higher request_timeout value, but I guess that is something you don't want to do.

Answer 2

I'm not sure if this is exactly what you're looking for, but may be helpful. Whenever I'm curious about what my cluster is doing, I check out the tasks API .

The tasks API shows you all of the tasks that are currently running on your cluster. It will give you information about individual tasks, such as the task ID, start time, and running time. Here's the command:

curl -XGET http://localhost:9200/_tasks?group_by=parents | python -m json.tool

Answer 3

just a side note hint, of why your requests might take a lot of time (unless you are just indexing too many in a single bulk run). If you have configured your own precision for geo shapes, also make sure you are configuring distance_error_pct , otherwise no error is assumed, resulting in documents with a lot of terms that take a lot of time indexing.