Is there a way to maintain aging in documents in elastic search

Question

Here is the problem

I have about 1 million record in indexes. There is a property aging in the documents which increase daily. Every night scheduler runs and it calculates the aging from current date and created date in the document and update the index.

The problem is as data is increasing the bulk update is leading to GC overhead limit exceeded . So what I did is added some pause in each update, but still no help.

Now I am thinking and researching of using groovy script with 'update_with_query'.

I want to ask it there any other way to maintain age. eg in jira everyday overdue date is increased or I have to fetch visit and update documents

EveryTime bulk request is run I can see elastic search throttling ' now throttling indexing: numMergesInFlight=5, maxNumMerges=4'. I have read about this but not sure what to do. I think there should be another approach to calculate aging but not sure, because as data will increase this problem is going to persist

IN the end I want a query like give me all docs whose aging is 100 or give me all documents whose aging > 100

Answer 1

The answer was simple. I was thinking other way around. if a query is get all docs where aging is > 2 . It means I need to get all docs who were created before two days. Simple convert '2' to date from current date and use range operation and it should solve the problem