How to update an existing document inside an ElasticSearch index using NEST?

Question

im attempting to do an index update of documents within my elasticsearch index.

a job is run periodically during the day that identifies database records that have been updated since the last time the job ran. i want to be able to update these specific records in the index. any of the fields may have changed in the record.

so i populate a dataset and then loop through the records to populate an instance of my class with all the properties from the database.

each time i want to update the corresponding record in the index or add it if it doesnt currently exist...

within my loop im trying some code like this to do the update...

client.Update<MyContentClass>(u => u
                .Id("AU-7zORce3_kxnyDoVLv")
                .Index("qubecontent")
                //.Doc(new MyContentClass { ESUniqueKey = MyContentClassInstance.ESUniqueKey })
                .DocAsUpsert()
                .Refresh()
                );

im not sure what Id is referencing? is this the id that elasticsearch autogenerates for each indexed record? I do generate an additional unique id within my class but not sure how i reference this?

Can anyone advise how i would perform this index update for a changed record?

Answer 1

The ID field in the upsert is indeed referencing the internal ElasticSearch ID (think of it as the primary key for ES). If you already have your own unique primary key you can use that as the primary key in ES as well. Take these examples:

Example 1: Let ES generate it's own ID

POST test/type1
{
  "f1": "record 1",
  "f2": "2000-01-01"
}

Result 1:

{
   "_index": "test",
   "_type": "type1",
   "_id": "AU--Dz-Kl6g2APRJ9y7l",
   "_version": 1,
   "created": true
}

You can see that ES generated it's own primary key of "AU--Dz-Kl6g2APRJ9y7l"

Example 2: Epecify your own ID

POST test/type1/thisIsMyID
{
  "f1": "record 1",
  "f2": "2000-01-01"
}

Result 2:

{
   "_index": "test",
   "_type": "type1",
   "_id": "thisIsMyID",
   "_version": 1,
   "created": true
}

Notice how in Example 2 it used the ID that I specified. Once you are using the same primary key as ES then you can run your upsert statement.

NOTE: If you are regenerating the whole document and what you really want to do is overwrite the old vs upsert the old. Then you can just POST again with the same ID and it will overwrite the old record with the new record. Overwriting will be WAY faster than upserting.