How to bulk insert Json using NEST elasticsearch?

Question

I'am trying to insert multiple records into my database using Nest. Inserting using IndexMany class does work however I also need to insert objects by json string.

I did look on github, and found some examples how to use the RAWclient. Below a code example I insert my json.

    > var twitter = _jsonData;          
    > var result = client.Raw.BulkPost(
    >               new { twitter }
    >               , qs => qs
    >                   //.Replication(ReplicationOptions.Async)
    >                   .Refresh(true)          );

some additional info:

jsondata:

tweet tweet1 = new tweet { id = "104", name = "test104", lastname = "test107" }; //ect....
List<tweet> data; //multiple tweet objects are added
string json = Newtonsoft.Json.JsonConvert.SerializeObject(data);

var twitter:

{
      "twitter": "[{'name':'test104','lastname':'test107','id':'104'},{'name':'test105','lastname':'test108','id':'105'},{'name':'test106','lastname':'test109','id':'106'}]"
}

result i receive from the database:

{"error":"Unexpected end-of-input: expected close marker for OBJECT (from [Source: [B@10893e4; line: 1, column: 0])\n at [Source: [B@10893e4; line: 2, column: 3]"}

Does anyone know what the issue might be? or what I'am missing in my json/code snipped?

Answer 1

Your json is not correct for elasticsearch bulk operation. See the documentation .

In a bulk request every data object should be preceded by a command because a single bulk request can contain inserts, updates or deletes, not just inserts. So your json should look like

  { "index" : { "_index" : "twitter", "_type" : "tweets" } }\n
  {'name':'test104','lastname':'test107','id':'104'}\n
  { "index" : { "_index" : "twitter", "_type" : "tweets" } }\n
  {'name':'test105','lastname':'test108','id':'105'}\n
  { "index" : { "_index" : "twitter", "_type" : "tweets" } }\n
  {'name':'test106','lastname':'test109','id':'106'}\n

To reduce overhead from repetitive commands you can move some arguments to the request uri. Then the json can be shorter:

  { "index" : { } }\n
  {'name':'test104','lastname':'test107','id':'104'}\n

In IRawElasticClient that means moving them to BulkPost arguments.

  var result = client.Raw.BulkPost(new { twitter }, "twitter", "tweets");