Need help in creating dynamic elasticsearch index using Java high level rest client bulk API

Question

I am creating an Elasticsearch cluster that will integrate with our Java codebase. I want to create an Elasticsearch index and insert SQL Query data into it from multiple databases. The query results from all the databases should be inserted into the same index. For that purpose I am using Java High level Rest Client. But I am not quite sure how to do this because a lot of methods from the old APIs are deprecated. I also am not quite so sure what to do with the createIndexResponse instance too. Can anyone help me in this?

        public static void method_1(Connection con) throws Exception {

            Statement statement = con.createStatement();
            try {
                ResultSet result = statement.executeQuery("SELECT Field_1, Field_2, Field_3 from Table_1");
                int counter = 1;
                CreateIndexRequest createIndexRequest = new CreateIndexRequest("index_name");
                createIndexRequest.settings(new Settings.Builder()
                        .put("cluster.name", "my_cluster")
                        .put("http.enabled", true)
                        .put("node.data", true)
                        .put("index.number_of_shards", 3)
                        .put("index.number_of_replicas", 1)
                        .build());
                CreateIndexResponse createIndexResponse = ElasticSearch.eclient.indices().create(createIndexRequest, RequestOptions.DEFAULT);
                BulkRequest bulkRequest = new BulkRequest();
                while (result.next()) {
                    String field_1 = result.getString("Field_1");
                    int field_2 = result.getInt("Field_2");
                    String field_3 = result.getString("Field_3");
                    XContentBuilder builder = XContentFactory.jsonBuilder()
                            .startObject()
                            .field("Field 1", field_1)
                            .field("Field 2", field_2)
                            .field("Field 3", field_3)
                            .endObject();
                    UpdateRequest updateRequest = new UpdateRequest("index_name", "_doc", Integer.toString(counter));
                    updateRequest.doc(builder);
                    bulkRequest.add(updateRequest);
                }
                BulkResponse response = ElasticSearch.eclient.bulk(bulkRequest, RequestOptions.DEFAULT);
                if (response.hasFailures()) {
                    for (BulkItemResponse item : response.getItems()) {
                        System.out.println(item.getFailureMessage());
                    }
                }
                counter++;
                statement.close();
            } catch (Exception ex) {
                ex.printStackTrace();
            }
        }

Answer 1

You did not give which version you are using. Anyway, I'm considering that you are using 7.0.0.

Here is an example which should be ok I guess:

try (RestHighLevelClient client = new RestHighLevelClient(
        RestClient.builder(HttpHost.create("http://localhost:9200")))) {
    CreateIndexRequest createIndexRequest = new CreateIndexRequest("index_name");
    createIndexRequest.settings(Settings.builder()
            .put("index.number_of_shards", 3)
            .put("index.number_of_replicas", 1)
            .build());
    client.indices().create(createIndexRequest, RequestOptions.DEFAULT);
    BulkRequest bulkRequest = new BulkRequest();
    XContentBuilder builder = XContentFactory.jsonBuilder()
            .startObject()
                .field("foo", "bar")
            .endObject();
    IndexRequest indexRequest = new IndexRequest("index_name");
    indexRequest.source(builder);
    bulkRequest.add(indexRequest);
    BulkResponse response = client.bulk(bulkRequest, RequestOptions.DEFAULT);
    if (response.hasFailures()) {
        for (BulkItemResponse item : response.getItems()) {
            System.out.println(item.getFailureMessage());
        }
    }
} catch (IOException e) {
    e.printStackTrace();
}

I'd recommend using the BulkProcessor class which is much easier to deal with IMO. I have a full demo repository which I'm updating on every new release. It might help you as well.