How to handle thousands of threads in Java without using the new java.util.concurrent package

Question

I have a situation in which I need to create thousands of instances of a class from third party API. Each new instance creates a new thread. I start getting OutOfMemoryError once threads are more than 1000. But my application requires creating 30,000 instances. Each instance is active all the time. The application is deployed on a 64 bit linux box with 8gb RAM and only 2 gb available to my application.

The way the third party library works, I cannot use the new Executor framework or thread pooling.

So how can I solve this problem?

Note that using thread pool is not an option. All threads are running all the time to capture events.

Sine memory size on the linux box is not in my control but if I had the choice to have 25GB available to my application in a 32GB system, would that solve my problem or JVM would still choke up?

Are there some optimal Java settings for the above scenario ?

The system uses Oracle Java 1.6 64 bit.

Answer 1

I'd say give up now and figure another way to do it. Default stack size is 512K. At 30k threads, that's 15G in stack space alone. To fit into 2G, you'll need to cut it down to less than 64K stacks, and that leaves you with zero memory for the heap, including all the Thread objects, or the JVM itself.

And that's just the most obvious problem you're likely to run into when running that many simultaneous threads in one JVM.

Answer 2

I concur with Ryan's Answer. But the problem is worse than his analysis suggests.

Hotspot JVMs have a hard-wired minimum stack size - 128k for Java 6 and 160k for Java 7.

That means that even if you set the stack size to the smallest possible value, you'd need to use roughly twice your allocated space ... just for thread stacks.

In addition, having 30k native threads is liable to cause problems on some operating systems.

I put it to you that your task is impossible. You need to find an alternative design that does not require you to have 30k threads simultaneously. Alternatively, you need a much larger machine to run the application.

---

Reference: http://mail.openjdk.java.net/pipermail/hotspot-runtime-dev/2012-June/003867.html

Answer 3

I think we are missing lots of details, but would a distributed plateform would work? Each of individual instances would manage a range of your classes instances. Those plateform could be running on different pcs or virtual machines and communicate with each other?

Answer 4

I had the same problem with an SNMP provider that required a thread for each outstanding get (I wanted to have tens of thousands of outstanding gets going on at once). Now that NIO exists I'd just rewrite the library myself if I had to do this again.

You cannot solve it in "Java Code" or configuration. Windows chokes at around 2-3000 threads in my experience (this may have changed in later versions). When I was doing this I surprisingly found that Linux supported even less threads (around 1000).

When the system stops supplying threads, "Out of Memory" is the exception you should expect to see--so I'm sure that's it--I started getting this exception long before I ran out of memory. Perhaps you could hack linux somehow to support more, but I have no idea how.

Using the concurrent package will not help here. If you could switch over to "Green" threads it might, but that might take recompiling the JVM (it would be nice if it was available as a command line switch, but I really don't think it is).