Lucene search stops working

Question

I am using lucene for search in one of my projects. It's running as a separate service on a port. Whenever a query comes, a request is sent to this server and it returns a map of results.

My problem is that it stops working after some time. It works fine for 1 day or so. But after 1 day it stops returning results (ie service is running but it results 0 results). To get it back working, I have to restart the service and then it starts working fine again.

Please suggest some solution. I'll be happy to provide more info if needed.

Thanks.

Answer 1

Were I to make a guess at an easy mistake to make that could cause such behavior, maybe you're opening a bunch of indexwriters or indexreaders as time goes on, and not closing them correctly, thus running out of file descriptors available on your server. See if the 'lsof' shows a lot of open descriptors on '.cfs', '.fdx' and/or '.fdt' ('ulimit -n' can be used to see the maximum).

One thing to note about the IndexSearcher, which I've seen cause problems:

Closing the searcher may not close the underlying reader. If you pass a reader into the searcher, it won't be closed when you close the searcher (since in that case, it may be in use by other objects). An example of this:

//Assume I have an IndexWriter named indexwriter, which I reuse.
IndexSearcher searcher = new IndexSearcher(IndexReader.open(indexwriter, true));

//Use the searcher

searcher.close();
//We close the search, but the underlying reader remains open.

Now this has accumlated an unclosed reader, and left some index file descriptors open. If this pattern is used enough times, over time it will stop responding. That one example of such an error anyway.

It can be fixed by simply closing the reader when you close the searcher, something like: searcher.getIndexReader().close() . Better solutions could be found, though. Reusing the reader, which can be refreshed when index contents change, for instance.

Don't know if this is the exact problem you are having, but might be worth noting.