Data has been deleted automatically when importing JOINed tables from MySQL

Question

I have joined 2 mysql tables by JOIN query and have imported data by DIH in SOLR.

But, Half of records deleted automatically. I come to know about this by reading numDocs , maxDocs & deletedDocs values.

I have checked all fields in tables that I have already defined in data-config.xml as well as in schema.xml files.

What is the problem in this?

Help me in this please.

Answer 1

Its a replace of an existing document

numDocs+DeletedDocs = MaxDocs 
 (821+1160=1981)

numDocs represents the number of searchable documents in the index maxDoc may be larger as the numDoc count as it includes logically deleted documents that have not yet been removed from the index.

If you reindex/repost numDocs will never increase, because the new documents will constantly be replacing the old...try optimizing the index, it might remove the logically deleted documents/indexs.

The Unique Key Field

<uniqueKey></uniqueKey>

The <uniqueKey> declaration can be used to inform Solr that there is a field in your index which should be unique for all documents. If a document is added that contains the same value for this field as an existing document, the old document will be deleted.