Bulk operations in Java Rest Client in Elasticsearch

Question

I am working with Java Rest client for elastic search https://www.elastic.co/guide/en/elasticsearch/client/java-rest/current/index.html but could not find a way to do bulk inserts or updates. How can I bulk operate with this client?

Answer 1

The 5.2 Java Rest client for Elasticsearch is String based and can become messy really quick. This is especially true for Bulk operations, since they are constructed from chaining JSON objects.

If you want / have to connect to your Elasticsearch cluster via REST-client, I recommend to useJEST client instead.

Here is an example on how to use the JEST Client for Bulk requests:

// Construct a new Jest client according to configuration via factory
JestClientFactory factory = new JestClientFactory();
factory.setHttpClientConfig(new HttpClientConfig
                       .Builder("http://localhost:9200")
                       .multiThreaded(true)
                       .build());
JestClient client = factory.getObject();

// Construct Bulk request from articles
Bulk bulk = new Bulk.Builder()
                .defaultIndex("twitter")
                .defaultType("tweet")
                .addAction(Arrays.asList(
                    new Index.Builder(article1).build(),
                    new Index.Builder(article2).build()))
                .build();

client.execute(bulk);

Answer 2

If you are using Java to work with your Elasticsearch Server, i would suggest you using Java API instead. Here is where you can take it: https://www.elastic.co/guide/en/elasticsearch/client/java-api/current/index.html

You can find how to do the bulk operation in Document API/Bulk API.

If you still need to use Java Rest client for some reason, you will need to build a payload in Elasticsearch's Bulk request format to be able to perform the request.

Please find out how to build the Bulk request format here: https://www.elastic.co/guide/en/elasticsearch/reference/current/docs-bulk.html (Basically, it's constructed from a list of json object)

Answer 3

I'm using data from .json file.

Elasticsearch version: 6.8.0

pom.xml:

<dependencies>
    <dependency>
        <groupId>org.elasticsearch</groupId>
        <artifactId>elasticsearch</artifactId>
        <version>6.8.0</version>
    </dependency>

    <dependency>
        <groupId>org.elasticsearch.client</groupId>
        <artifactId>transport</artifactId>
        <version>6.8.0</version>
    </dependency>

    <dependency>
        <groupId>commons-io</groupId>
        <artifactId>commons-io</artifactId>
        <version>2.1</version>
        <scope>test</scope>
    </dependency>
</dependencies>`

Java

String bulkContent = new String(Files.readAllBytes(new File(filePath).toPath())); HttpEntity entity = new NStringEntity(bulkContent, ContentType.APPLICATION_JSON); Request request = createRequest(indexName, indexType, httpMethod, entity); RestClient restClient = RestClient.builder(new HttpHost(hostname, port, scheme)).build(); Response response = restClient.performRequest(request);