Using ThreadPool excutor service in tomcat to speed up requests

Question

I have a tomcat6 servlet application. One of my requests (~10 seconds avg.) can significantly be improved by using multi threading because it is an CPU-only task and I have >= 8 cores. I just wonder if it is clever to do so or just a cosmetic change:

For the single user case it is an improvement, of course. But what happens if the load increases? I have a finite amount of CPU power which is shared among several HTTP connector threads at the moment. Assuming I had configured them optimally, I would have to take some threads our of the http connector thread pool and put it into some executor server in order to speed up this single (but important) operation.

My assumption is, that with increasing load, my system will perform worse if I use an additional threaded executor service.

Do you see my problem? Does anyone have some best-practise ideas? Or something I overlooked?

Answer 1

In cases of performance questions, with few exceptions, the best answer is usually to formulate a benchmarking test and just try it.

Keep in mind that some tasks can't be parallelized. That is, attempting to do so either requires synchronization such that no benefit is gained or is simply not possible at all as each step requires the previous to be completed. If your task can not be parallelized then there will be no benefit.

On that same token, not all of your application's activities may necessarily run in parallel. To some extent portions of your application will block each other for I/O either to the file system or to the network, and even possibly to an extent within your database waiting for requests. All of which means that just because your hardware may only have say 8 core (for example) doesn't mean you should strictly limit yourself to 8 or 9 Threads. Of course, you don't want to go crazy and have hundreds either.

Answer 2

As I understand the problem, you want to create few extra threads which will benefit if, say, you have 1 concurrent request and you split your work (which takes ~10 seconds) into few smaller work units which can be parallelized and later joined.

And you are worried this can actually degrade the performance if, for example, you have 100 concurrent requests as you don't have spare cores to parallelize each of these 100 jobs.

Theoretically the smallest overhead is when the number of active threads is equal to number of physical cores. So you need to ask yourself a question - what is your most common case (how many users does the system have?) and what price are you ready to pay if number of users peak.

Anyway, I completely agree with Tim that you should benchmark, to judge this theoretically is virtually impossible. For example, your results can be totally different if your 10 second task is 100% CPU-bound VS 80% CPU-bound. Measure, don't guess .