How to optimize number of threads to speed up processing

Question

I have read many similar questions . However I was not quite satisfied with answers.

I would like to build an algorithm that would adjust the number of threads depending on the average speed .

Let's say as I introduce a new thread, the average speed of task execution increases , it means that the new thread is good. Then the algorithm should try to add another thread ... until the optimal number of threads is achieved .......

Also the algorithm should be keeping track of the average speed. If at some point the average speed goes down significantly, let's say by 10 % (for any reason eg i open a different application or whatever) , then the algorithm should terminate one thread and see if the speed goes up ...

Maybe such an API exists. Please, give me any directions or any code example how I could implement such an algorithm

Thank You !

Answer 1

I do not know self-tune system that you are describing but it sounds like not so complicated task once you are using ready thread pool. Take thread pool from concurrency package, implement class TimeConsumptionCallable implements Callable that wraps any other callable and just measures the execution time.

Now you just have to change (increase or decrease) number of working threads when average execution time increases or decreases.

Just do not forget that you need enough statistics before you decide to change number of working threads. Otherwise various random effects that do not depend on your application can cause your thread pool to grow and go down all the time that can itself kill overall performance.

Answer 2

newCachedThreadPool() V/s newFixedThreadPool suggests that perhaps you should be looking at ExecutorService.newCachedThreadPool() .

Creates a thread pool that creates new threads as needed , but will reuse previously constructed threads when they are available . These pools will typically improve the performance of programs that execute many short-lived asynchronous tasks. Calls to execute will reuse previously constructed threads if available. If no existing thread is available, a new thread will be created and added to the pool. Threads that have not been used for sixty seconds are terminated and removed from the cache. Thus, a pool that remains idle for long enough will not consume any resources. Note that pools with similar properties but different details (for example, timeout parameters) may be created using ThreadPoolExecutor constructors.

Answer 3

If your threads do not block at any time, then the maximum execution speed is reached when you have as many threads as cores, as simply more than 100% CPU usage is not possible.

In other situations it is very difficult to measure how much a new thread will increase/decrease the execution speed, as you just watch a moment in time and make assumptions based on something that could be entirely different the next second.

One idea would be to use an Executor class in combination with a Queue that you specified. So you can measure the size of the queue and make assumptions based on that. If the queue is empty, threads are idle and you can remove one. If the queue fills up, threads cannot handle the load, you need to add more. If the queue is stable, you are about right.

Answer 4

You can come up with your own algorithm by using existing API of java :

public void setCorePoolSize(int corePoolSize) in ThreadPoolExecutor

Sets the core number of threads. This overrides any value set in the constructor.

If the new value is smaller than the current value, excess existing threads will be terminated when they next become idle.

If larger, new threads will, if needed, be started to execute any queued tasks.

Initialization:

ExecutorService service = Executors.newFixedThreadPool(5); // initializaiton

On your need basis, resize the pool by using below API

((ThreadPoolExecutor)service).setCorePoolSize(newLimit);//newLimit is new size of the pool 

And one important point: If the queue is full, and new value of number of threads is greater than or equal to maxPoolSize defined earlier, Task will be rejected.

Be careful when setting maxPoolSize so that setCorePoolSize works properly.