Elasticsearch批量索引没有错误

Question

I am running the code below on the cmd line (Windows) to bulk index about 3 million docs, but nothing happens. I run it and one second later it is done with no output and the index is not even created.

 curl -H "Content-Type: application/json" -XPOST "localhost:9200/test1/_doc/_bulk" --data-binary "@huge_JSON"

I run the same line on a few lines of this document and it works fine.

Also running it on a file with 300k documents works.

Why is it not working and not giving an error?

Edit: For whatever reason I now get the error:

curl: option --data-binary: out of memory

Answer 1

Split up the file (Use cygwin etc. if on Windows):

split -l 100000 huge_JSON

Run this line to index each of the split files:

for %%i in (*) do curl -H "Content-Type: application/json" -XPOST "localhost:9200/test1/_doc/_bulk?pretty&refresh" --data-binary  @%%i

OR

Just use logstash