Best way to reuse a Runnable

Question

I have a class that implements Runnable and am currently using an Executor as my thread pool to run tasks (indexing documents into Lucene).

executor.execute(new LuceneDocIndexer(doc, writer));

My issue is that my Runnable class creates many Lucene Field objects and I would rather reuse them then create new ones every call. What's the best way to reuse these objects ( Field objects are not thread safe so I cannot simple make them static) - should I create my own ThreadFactory ? I notice that after a while the program starts to degrade drastically and the only thing I can think of is it's GC overhead. I am currently trying to profile the project to be sure this is even an issue - but for now lets just assume it is.

Answer 1

Your question asks how to reuse a Runnable, so I am going to ignore the other details adn simply answer that question.

If you are using a ThreadPoolExecutor, you can use the [ThreadPoolExecutor#afterExecute][1] method to return the Runnable object to a pool/queue of 'cached' Runnables.

[1]: http://java.sun.com/javase/6/docs/api/java/util/concurrent/ThreadPoolExecutor.html#afterExecute(java.lang.Runnable , java.lang.Throwable)

Answer 2

A Runnable object is reusable. It is thread object which is not.

Best way ? it is your way :-)

I think it is more a lucene question than a runnable question.

Answer 3

You might want to do some more benchmarking to nail down what's causing your slowdowns.

I'm willing to bet that your problem is not related to the creation of Field instances. Field doesn't have any finalizers, and they're not designed to be pooled.

Answer 4

For now I have decided to just use a simple Producer->Consumer model. I pass a BlockingQueue to each indexer, rather then a document to index, and then have the main driver of the program add new documents to that queue. The Indexers then feed off that [bounded] queue and reuse the Field objects and share the thread-safe IndexWriter .

I did find a place where I was possibly not calling HttpMethod.releaseConnection() so that could have caused my memory issues (uncertain).