Spring Boot: gracefully shutdown by controlling termination order involving MongoClient

Question

I have a Spring Boot application that spawns many threads using an AsyncTaskExecutor (the number is predefined)

The threads execute an infinite loop, that reads from some queue and process objects, so I don't really have a rejection policy mechanism (like a ThreadPool that accept tasks )

The problem is that when the application gets closed, threads might (and probably) be busy with processing an item which includes operations to Mongo using MongoTemplate .

So when the application gets closed the MongoClient is being close() 'd automatically and then I'm getting some errors from Mongo, like:

java.lang.IllegalStateException: The pool is closed
    at com.mongodb.internal.connection.ConcurrentPool.get(ConcurrentPool.java:137)
    at com.mongodb.internal.connection.DefaultConnectionPool.getPooledConnection(DefaultConnectionPool.java:262)
    at com.mongodb.internal.connection.DefaultConnectionPool.get(DefaultConnectionPool.java:103)
    at com.mongodb.internal.connection.DefaultConnectionPool.get(DefaultConnectionPool.java:92)
    at com.mongodb.internal.connection.DefaultServer.getConnection(DefaultServer.java:85)

How can I close the application gracefully? eg interrupting the threads while not shutting down the MongoClient just yet?

CODE:

Bean creation:

@Bean
AsyncTaskExecutor getTaskExecutor() {
    SimpleAsyncTaskExecutor executor = new SimpleAsyncTaskExecutor();
    return executor;
}

Execution simply by:

executor.execute(runnable);

Answer 1

Don't use SimpleAsyncTaskExecutor - SimpleAsyncTaskExecutor creates a new thread for each request instead use ThreadPoolTaskExecutor and configure two properties noted below.

/**
 * Set whether to wait for scheduled tasks to complete on shutdown,
 * not interrupting running tasks and executing all tasks in the queue.
 * <p>Default is "false", shutting down immediately through interrupting
 * ongoing tasks and clearing the queue. Switch this flag to "true" if you
 * prefer fully completed tasks at the expense of a longer shutdown phase.
 * <p>Note that Spring's container shutdown continues while ongoing tasks
 * are being completed. If you want this executor to block and wait for the
 * termination of tasks before the rest of the container continues to shut
 * down - e.g. in order to keep up other resources that your tasks may need -,
 * set the {@link #setAwaitTerminationSeconds "awaitTerminationSeconds"}
 * property instead of or in addition to this property.
 * @see java.util.concurrent.ExecutorService#shutdown()
 * @see java.util.concurrent.ExecutorService#shutdownNow()
 */
public void setWaitForTasksToCompleteOnShutdown(boolean waitForJobsToCompleteOnShutdown) {
    this.waitForTasksToCompleteOnShutdown = waitForJobsToCompleteOnShutdown;
}

/**
 * Set the maximum number of seconds that this executor is supposed to block
 * on shutdown in order to wait for remaining tasks to complete their execution
 * before the rest of the container continues to shut down. This is particularly
 * useful if your remaining tasks are likely to need access to other resources
 * that are also managed by the container.
 * <p>By default, this executor won't wait for the termination of tasks at all.
 * It will either shut down immediately, interrupting ongoing tasks and clearing
 * the remaining task queue - or, if the
 * {@link #setWaitForTasksToCompleteOnShutdown "waitForTasksToCompleteOnShutdown"}
 * flag has been set to {@code true}, it will continue to fully execute all
 * ongoing tasks as well as all remaining tasks in the queue, in parallel to
 * the rest of the container shutting down.
 * <p>In either case, if you specify an await-termination period using this property,
 * this executor will wait for the given time (max) for the termination of tasks.
 * As a rule of thumb, specify a significantly higher timeout here if you set
 * "waitForTasksToCompleteOnShutdown" to {@code true} at the same time,
 * since all remaining tasks in the queue will still get executed - in contrast
 * to the default shutdown behavior where it's just about waiting for currently
 * executing tasks that aren't reacting to thread interruption.
 * @see java.util.concurrent.ExecutorService#shutdown()
 * @see java.util.concurrent.ExecutorService#awaitTermination
 */
public void setAwaitTerminationSeconds(int awaitTerminationSeconds) {
    this.awaitTerminationSeconds = awaitTerminationSeconds;
}

Relavant part

Set the maximum number of seconds that this executor is supposed to block on shutdown in order to wait for remaining tasks to complete their execution before the rest of the container continues to shut down. This is particularly useful if your remaining tasks are likely to need access to other resources that are also managed by the container.

You can configure using spring auto configuration to control the task execution properties ( preferred ) or programmatically with @Bean annotation

Spring boot in 2.1.0 provides auto configuration for task executors and uses for both @EnableAsync and Spring MVC Async support.

There is no task executor bean / webMvcConfigurer configuration is needed from application. So remove the one you have and should be good.

You can adjust using applicaion properties/yml file with spring.task.execution.* .

spring.task.execution.shutdown.await-termination=true
spring.task.execution.shutdown.await-termination-period=60

Full listing can be found here

More details here and here

OR

@Bean
public ThreadPoolTaskExecutor threadPoolTaskExecutor() {
    ThreadPoolTaskExecutor taskExecutor = new ThreadPoolTaskExecutor();
    taskExecutor.setCorePoolSize(5);
    taskExecutor.setMaxPoolSize(5); 
    taskExecutor.waitForTasksToCompleteOnShutdown(true);
    taskExecutor.setAwaitTerminationSeconds(60);
    return taskExecutor;
  }