为什么我的Java长时间运行的线程（5k +线程）没有利用所有机器内核（12个内核）？

Question

I've Worte a simple multithread java application, The main method just creates 5k threads, each thread will loop over a list having 5M records to process.

My Machine specs:

• CPU cores: 12 cores
• Memory: 13Gb RAM
• OS: Debian 64-bit

My jar is now running, And I use hTop to monitor my application and this is what I can see while its running

And This is how I construct a Thread:

ExecutorService executor = Executors.newCachedThreadPool();
Future<MatchResult> future = executor.submit(() -> {
            Match match = new Match();
            return match.find(this);
        });

Match.class

find(Main main){
// looping over a list of 5M 
// process this values and doing some calculations 
// send the result back to the caller 
// this function has no problem and it just takes a long time to run (~160 min)
}

And now I have some questions:

1- Based on my understanding if I have a multiThreaded process, it'll fully utilize all my cores until the task is completed, so why the work load is only around 0.5 (only half a core is used)?

2- Why my Java app state is "S" (sleeping) while its actually running and filling up the logs file?

3- Why I can only see 2037 threads out of 5k are running (this number was actually less than this and its getting increased over time)

My Target: to utilize all cores and get all this 5k+ done as fast as it can be :)

Answer 1

Based on my understanding if I have a multiThreaded process, it'll fully utilize all my cores until the task is completed.

Your understanding is not correct. There are lots of reasons why cores may not (all) be used in a poorly designed multi-threaded application.

so why the work load is only around 0.5 (only half a core is used)?

A number of possible reasons:

1. The threads may be deadlocked.
2. The threads may all be contending for a single lock (or a small number of locks), resulting in most of them waiting.
3. The threads could all be waiting for I/O; eg reading the records from some database.

And those are just some of the more obvious possible reasons.

Given the that your threads are making some progress, I think that explanation #2 is a good fit to your "symptoms".

---

For what it is worth, creating 5k threads is almost certainly a really bad idea. At most 12 of those could possibly be running at any time. The rest will waiting to run (assuming you resolve the problem that is leading to thread starvation) and tying down memory. The latter has various secondary performance effects.

My Target: to utilize all cores and get all this 5k+ done as fast as it can be :)

Those two goals are probably mutually exclusive :-)

---

All threads are logging to the same file by a the java.util.Logger.

That is a possibly leading to them all contending for the same lock ... on a something in the logger framework. Or bottlenecking on file I/O for the log file.

Generally speaking logging is expensive. If you want performance, minimize your logging, and for cases where the logging is essential, use a logging framework that doen't introduce a concurrency bottleneck.

---

The best way to solve this problem is to profile the code and figure ouot where it is spending most of its time.

Guesswork is inefficient.

Answer 2

Thank you guys, I've fixed the problem and now Im having the 12 cores running up to maximum as you see in the picture. :)

I actually tried to run this command jstack <Pid> to see the status of my all running threads in this process ID, and I found that 95% of my threads are actually BLOCKED at the logging line, I did some googling and found that I can use AsynchAppender in log4J so logging will not block the thread