从Elasticsearch群集的索引中获取所有文档，然后在另一个ElasticSearch群集中对其进行索引

Question

My goal here is to get all documents from an index of an ES cluster and insert them in another ES cluster keeping the same metadata.

I had a look at mget API to retrieve data and Bulk API to insert it but this Bulk API needs a special structure:

action_and_meta_data\n
optional_source\n
action_and_meta_data\n
optional_source\n

So my idea is to retrieve my data of EScluster1 in a file and rearranged it to meet the structure of Bulk API and index it to EScluster2.

Do you see a better and/or faster way to proceed?

Answer 1

elasticdump does this. If you want to do this manually, you'll want to query using scroll and then bulk index what comes out of that. Not too hard to script together. With elastic dump you can pump the data around without writing to a file. However, it is kind of limited when you have eg parent/child relations in your index.