How to configure a ThreadPool for parallel webservice requests?

Question

I want to send webservice calls to a webservice in parallel. There should be a maximum of 20 parallel requests waiting for a webservice response. Any other requests should wait for them to finish.

If a single user sends a request to me, this usually leads to sending 5 parallel requests to the target server. So I could serve a maximum of 20/5 = 4 users at a time instantly. Others would have to wait, which is fine. Or being rejected on really high load.

Question: which thread pool should I use for this, and how should I configure it?

private ExecutorService asyncExecutor() {
    ThreadPoolTaskExecutor executor = new ThreadPoolTaskExecutor();
    executor.setCorePoolSize(20);
    executor.setMaxPoolSize(20);
    executor.setQueueCapacity(100);
    executor.initialize();

    //return executor.getThreadPoolExecutor(); //I could as well use this. Which is better??
    return Executors.newFixedThreadPool(20, executor);
}

I read the above as follows: the main pool can send 20 parallel requests to the webserver (thereby serve 4 users instantly in parralel). If all 20 threads are busy and a 5th user comes in, the requests are queued.

Question:

• It the configuration correct?

• Why do I have to set the nthreads=20 in Executors.newFixedThreadPool() additionally? What is the difference in setting the poolSize ?

• Is there a performance overhead of setting the corePoolSize=20 ? I mean, then the pool will never thrink. But I could probably not set it lower (eg to 5), because new threads are only created if the queue is exhausted. Right?

Answer 1

I think that

executor.setMaxPoolSize(20) 

can be used to override initially set # of threads via constructor, it seems redundant in this case. Same applies to setCorePoolSize . It's just for dynamic control. As per size of the pool, in general you may consider Little's Law

Answer 2

When calling Executors.newFixedThreadPool(20, executor); you are passing previously set up executor as a ThreadFactory , because ThreadPoolTaskExecutor implements ThreadFactory interface.

The result is an ExecutorService that starts with 20 threads ready, maximum number of threads 20 and an unbounded LinkedBlockingQueue for tasks waiting to be processed (you can see it here https://docs.oracle.com/javase/7/docs/api/java/util/concurrent/Executors.html#newFixedThreadPool(int,%20java.util.concurrent.ThreadFactory) ).

So as to your first and second question- you can omit Executors.newFixedThreadPool() call and just return ThreadPoolTaskExecutor or call getThreadPoolExecutor() on it if you want to work with jdk class ThreadPoolExecutor.

As to your thrid question- there will be a penalty at a startup because jvm will have to create and run 20 threads but I guess thats negligible. One thing that will give you setting corePoolSize to something lower than maximumPoolSize is that unsed threads will be killed by jvm which can reduce the memory footprint of your application.

As to when new threads are created in ExecutorService- threads in ExecutorService are created when there are tasks in the queue and maximumPoolSize is not reached. If maximumPoolSize in executor is reached and queue is full- executor will throw RejectedExecutionException for every new task submitted to it, untill there is room again for a new ones. You can change this behaviour by implementing your own RejectedExecutionHandler.