Elasticsearch bulk insert using rest client

Question

To improve performance, I want to send documents to Elasticsearch in bulk instead sending one by one. I've read about elastic bulk API at https://www.elastic.co/guide/en/elasticsearch/client/java-api/current/java-docs-bulk.html

However, I am using Elasticsearch rest-client ( https://www.elastic.co/guide/en/elasticsearch/client/java-rest/current/index.html ) and couldn't find any example or documentation about how to make a bulk insert. All I could find was about bulk requests through transport client.

I guess I have to prepare request body as described here ( https://www.elastic.co/guide/en/elasticsearch/reference/current/docs-bulk.html ) and pass it to restclient's performRequest method? Is there another way, for instance, a builder mechanism in the ES java rest-client library, to make bulk insert using rest?

Answer 1

Yes, that's correct, for now the REST client only allows to send raw REST queries to ES but nothing too sophisticated. Elastic is working on a high-level client next that will work on top of the REST client and allow you to send DSL queries, etc.

For now, here is a sample code that you can use to send documents in bulk to your ES server:

RestClient client = ...;
String actionMetaData = String.format("{ \"index\" : { \"_index\" : \"%s\", \"_type\" : \"%s\" } }%n", index, type);

List<String> bulkData = ...; // a list of your documents in JSON strings    
StringBuilder bulkRequestBody = new StringBuilder();
for (String bulkItem : bulkData) {
    bulkRequestBody.append(actionMetaData);
    bulkRequestBody.append(bulkItem);
    bulkRequestBody.append("\n");
}
HttpEntity entity = new NStringEntity(bulkRequestBody.toString(), ContentType.APPLICATION_JSON);
try {
    Response response = client.performRequest("POST", "/your_index/your_type/_bulk", Collections.emptyMap(), entity);
    return response.getStatusLine().getStatusCode() == HttpStatus.SC_OK;
} catch (Exception e) {
    // do something
}

Answer 2

Another example in addition to Val answer: http://web.archive.org/web/20180813044955/http://cscengineer.net/2016/10/22/elastic-search-bulk-api/

Just use POST instead of PUT (be careful with the .exchange in the use of rest template)