lucene index file randomly crash and need to reindex

Question

How you all deal with such issues of occasionally need to reindex? what recommendation do you suggest to minimize this?

Answer 1

If you have a really large dataset, then i would recommend you to maintain a second server that has a backup of your index, so that you can have no single point of failure . You can synchronize this "backup" when ever you finish re-indexing and thus serve as a second search server that shares requests. If one fails, then the other can back-it-up. This is a simple yet robust approach to your problem and recommended for small-to-medium setups.