What is the best way to create thread pool in java

Question

I am trying to use executor services in one of my application where I have created a pool of 8 since my machine has 4 cores and as per my recent searches, I found that only 2 active threads can work on a core. When I checked number of cores via java also found the value to 4

int cores = Runtime.getRuntime().availableProcessors();
ExecutorService executor = Executors.newFixedThreadPool(cores*2);

Please suggest am I doing correctly because I dont see any much worth of creating a pool of 500 when my cpu can handle only 8 threads.

Answer 1

Hyper-threading

You should read up on hyper-threading technology. That term is specifically the brand-name used by Intel for its proprietary simultaneous multithreading (SMT) implementation, but is also used more generally for SMT.

An over-simplified explanation:

In a conventional CPU, switching between threads is quite expensive. Registers are the holding place for several pieces of data actually being worked on by the CPU core. The values in those registers have to be swapped out when switching threads. Likewise caches (holding places for data, slower than registers but faster than RAM) may also be cleared. All this takes time.

Hyper-threading design in a CPU adds a duplicate set of registers. Each core having a double set of registers means it can switch between threads without swapping out the values in the registers. The switch between threads is much faster, so much so that the CPU lies to the operating system, reporting each core as a pair of (virtual) cores. So a 4-core chip will appear as 8 cores, for example.

I found that only 2 active threads can work on a core

Be aware that switching threads still has some expense, just a much lower expense. A hyper-threaded CPU core is still executing only one thread at a time. Being hyper-threaded means the switching between threads is easier & faster.

For use an a machine where the threads are often in a holding pattern, waiting for some external function to complete such as a call out over the network, hyper-threading makes much sense. For applications where the cores are likely to be doing the kind of work that is CPU-bound such as number-crunching, simulations, scientific data analysis, then hyper-threading may not be as useful. So on machines doing such work, the sysadmin may decide to disable hyper-threading, so 4 cores are really just 4 cores for example. Also, because of the recent security vulnerabilities related to hyper-threading technology, some sysadmins may decide to disable hyper-threading.

Thread pools

creating a pool of 500 when my cpu can handle only 8 threads.

The sizing of a thread pool depends on the behavior of your application(s). If have CPU-bound apps, then you certainly want to limit the number of such CPU-intensive threads to less than the number of actual or virtual cores. If your apps are not CPU bound, if they are often doing file I/O , network I/O, or other activities where they are often do nothing while waiting on other resources, then you can have more threads than cores. If your threads often sit idle doing nothing at all, then you can have even more threads going.

What is the best way to create thread pool in java

There are no specific rules to help you here. You must make an educated guess initially, then monitor your app and host machine in production . And so you may want to have a way to set the number of threads being used in your apps during runtime rather than hard-coding a number. For example, use preference settings or use JMX . Learn to use profiling tools such as Java Flight Recorder and Mission Control ; both are now bundled with OpenJDK-based distributions of Java. If you are deploying to a system supporting DTrace (macOS, BSD, etc.), that may help as well.

Within an app with different kinds of workloads going on in various parts of functionality, it may make sense to maintain multiple thread pools. Use a pool with a very small number of threads for the CPU-intensive work, and a pool with a larger number of threads for CPU-non-intensive work. The Executors framework within modern Java helps make this easy.

Take into account all your apps you may be deploying to a machine. And take account of all the other apps running on that machine. And take account of the CPU needs of the operating system. After all this, you may find that some of your thread pools should be set to only one or two threads at most.

Tricky stuff

Thread-safety is very tricky complicated work. When sharing resources between threads (variables, files, etc.) you must educate yourself about the issues involved in protecting those resources from abuse.

Required reading: Java Concurrency in Practice by Brian Goetz et al.

Answer 2

Go to System properties and check how many cores (physical cores) and logical processors (virtual cores) are there.

For example :

if your system has n cores and n logical processors . This means your processor don't have a support for hyperthreading .

if your system has n cores and nx 2 logical processors . This means your processor have a support for hyperthreading . You can execute n * 2 threads in parallel.

Note : Suppose you have hyperthreading support. Now, you have 8 cores and 16 virtual cores.

Then, the processor will give a good throughput up to 16 threads. If you increase the thread pool more than 16 threads, the throughput will become uniform and will not change too much.