High thread count in java thread dump

Question

My java web application running on wildfly-9.0.1.Final sometimes it consumes very high memory and CPU. I got the thread dump for this time and found more than 7K thread with an identical thread with below stack trace.

pool-47940-thread-1 - priority:5 - threadId:0x000000029ad3f800 - nativeId:0xfad0 - state:WAITING
stackTrace:
java.lang.Thread.State: WAITING (parking)
at sun.misc.Unsafe.park(Native Method)
- parking to wait for <0x00000007a7d1fbe0> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1067)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:745)
Locked ownable synchronize

I analyze thread dump using this

Here is a screenshot for the same.

I used below code in my application many times

ExecutorService executor = Executors.newFixedThreadPool(3);
//business logic
// This will make the executor accept no new threads and finish all existing threads in the queue
executor.shutdown();
// Wait until all threads are finish
while (!executor.isTerminated()) {

}

Is ablow code cause any issue?

Are this threads root cause for high memory and CPU usage?

Answer 1

In my above code, I used while (!executor.isTerminated()){} to wait for all task finished their job. This while loop uses CPU until all task finished, this may cause high CPU issue.

I modified my code as below. This solved my high CPU problem.

ExecutorService executor = Executors.newFixedThreadPool(3);
executor.shutdown();
executor.invokeAll(listOfTask);  // invokAll method will wait untill all task finished, No while loop here

for more information read this

Answer 2

Huge amount of thread may cause problem with memory, but if all of them in WAITING state then they doesn't consume processor ticks.

I suggest you to solve your problems one by one.

To find out where all memory is stuck you need to get memory dump, and analyze it with some tool like MAT.

CPU consumption problem can by analyzed with jconsole or jvisualvm from SDK. Both of them have profiler tool.