Too many open files in Lucene Indexing when number of users increase

Question

I have been using Lucene Indexing and search technique for my web application. Initially I faced the "Too many Open files problem", after research I fixed it by using a common IndexSearcher for single user. Then, when I tested the application with a greater number of concurrent users, it again started to bring "Too many Open files problem".

I am keeping the IndexSearcher Object in the session of particular user, which is always keeping a few files open(which they are currently using). Could any one please help to overcome this issue even if there is any number of users?

Let me know if I am not clear anywhere in the above explanation.

Answer 1

Lucene's FAQ has a nice entry about it .

In a nutshell:

• Check that all your Lucene IndexReader, IndexWriter, and IndexSearcher are correctly closed
• Try to use compound files, that limits the number of files created
• Increase the limit of opened file handles on your operating system

I had similar problem with Lucene a while ago, and when you have a lot of concurrent users, this limit can easily be reached. Increasing the file handles limit worked perfectly fine.