如何通过在Java中的后请求中发送文件将数据导入Elasticsearch？

Question

Currently I am using following cmd to import data into elasticsearch

curl -XPOST 'localhost:9200/index/_bulk?pretty' --data-binary @required.json

ElasticSearch Bulk API

Now I am creating a java console application and I need to send this file in a post request. I am able to do this using SoapUI ie attaching a file with header

Content-Type: text/javascript

Doing this request from SOAPUI is successful and data is loaded.

---

I have looked into how to do a post request and I have got to following code. So my questions are :

1. How would we attach a file to the request ?

2. If file attachment is not possible, what is other possible solution ?

    URL url = new URL("http://localhost:9200/index/_bulk/"); HttpURLConnection httpCon = (HttpURLConnection) url.openConnection(); httpCon.setDoOutput(true); httpCon.setRequestMethod("POST"); httpCon.setRequestProperty("Content-Type", "text/javascript"); // (or text/plain) httpCon.setRequestProperty("Accept", "application/json"); OutputStreamWriter out = new OutputStreamWriter(httpCon.getOutputStream()); out.write(...filedata...); // <------ How to Put file data in output stream ? out.flush(); out.close(); 

Answer 1

How to write file contents to output

//out.write(...filedata...); // <------ How to Put file data in output stream ?

// Create a path to your file
Path path = Paths.get("D:/TEMP/bulk.json"); // <------ your file name

// Open a BufferedReader to your file - change CharSet if necessary
BufferedReader fReader = Files.newBufferedReader(path, Charset.defaultCharset());

String data = null;
// Read each line, append "\n" to it as required by Bulk API and write it to the server
while ((data = fReader.readLine()) != null) {
    out.write(data + "\n"); // <------ Put each line in output stream
    out.flush();            // <------ You may do this outside the loop once
}

out.close();

Note: There are multiple ways of writing file contents to an outputstream. I have just specified one. You may explore others.

bulk.json

{ "index" : { "_index" : "test", "_type" : "string", "_id" : "10" } }
{ "field1" : "value10" }
{ "index" : { "_index" : "test", "_type" : "string", "_id" : "20" } }
{ "field1" : "value20" }
{ "index" : { "_index" : "test", "_type" : "string", "_id" : "30" } }
{ "field1" : "value30" }

Test

http://localhost:9200/test/string/10
{"_index":"test","_type":"string","_id":"10","_version":3,"found":true,"_source":{ "field1" : "value10" }}

Also, since your client is Java, have you considered using the ElasticSearch Java Client API? I don't know it well but it should support Bulk requests. Look at the client.bulk(BulkRequest) method.