Limit on number of threads in Thread pool in Java executors framework?

Question

I have been working on Spring Boot application where I have limited memory for JVM upto 2 GB.

At controller, user sends some request which is handled by my Executors thread pool. It is a light weight task where each thread requires just few String variables of memory while its processing.

Considering I might deploy this application in PROD, Is there is a limit of user requests I can cater to with this approach.

What is the maximum limit? If no, will it be until JVM memory is full?

Thanks for the help.

Answer 1

You told us about your max memory but we still have no clue as of how much memory one thread would consume. Similarly you did not tell us how many CPU cores the application is allowed to use.

Depending on how your application uses compute resources different amount of parallel threads may become the optimal throughput. If the application shall coexist with other applications it may even not be the goal to use the compute resources to the maximum.

Therefore I advise to make the size of your threadpool configurable - be it a command line parameter, environment variable, configuration file or else. Some sysadmin can then tune the application based on the available resources and wished performance throughput.

Answer 2

Considering I might deploy this application in PROD, Is there is a limit of user requests I can cater to with this approach.

It seems like you are maximizing concurrency. There are many factors missing from your description. First is the config of spring. For springboot application with tomcat as the underlying serverlet container, the default max thread number is 200, with 10000 as its max connection number. Second is the heap size of JVM. More space for heap, less space for stack, which limit the thread number since every thread contains thread-local objects. Also, the CPU core information of deployed machine is missing.

To fully calculate the exact concurrency, it is better to do some performance tests with mocked requests.

Answer 3

Let's assume that you've setup your SpringBoot app server to accept incoming requests without any limits. If you assign 2 GB as your JVM heap size, then the concurrent requests that you can handle depends on -

• The memory taken up by each request
• Other JVM overhead (app server, spring and other libraries etc.,)
• How fast your threads complete their requests and free up resources

Once your heap space is full, the following requests would start throwing OutOfMemory exceptions.