Solr DIH - Small index still take time?

Question

This is the situation, I have two sources of data in my dataimport handler, one is huge, the other is tiny:

Source A: 10-20 records 
Source B: 50,000,000 records 

I was wondering what happens if I was to do a DIH just on Source A every 10 mins, and only run the DIH on source B every 24 hours.

Would running my DIH on Source A be extremely quick, because the data we are importing is small, or would it still be time consuming, because it would have to rebuild the index of the entire SOLR (ie 50,000,010 records).

Thank you!

Answer 1

Would running my DIH on Source A be extremely quick, because the data we are importing is small, or would it still be time consuming, because it would have to rebuild the index of the entire SOLR (ie 50,000,010 records).

Data Import from Source1 to your Index should be quick , provided you update only the 10 - 20 records rather than 50,000,010 records .

You could do so by providing entity name during dataimport as follows:

http://localhost:8080/solr/collection1/dataimport?command=full-import&entity=Source1

Even for these 10 - 20 records , you could do delta-import on subsequent calls, after doing an initial full-import .