单个 ScheduledExecutorService 实例与多个 ScheduledExecutorService 实例

Question

I have a service which schedules async tasks using ScheduledExecutorService for the user. Each user will trigger the service to schedule two tasks. (The 1st Task schedule the 2nd task with a fixed delay, such as 10 seconds interval) pseudocode code illustration:

task1Future = threadPoolTaskScheduler.schedule(task1);
for(int i = 0; i< 10000; ++i) {
    task2Future = threadPoolTaskScheduler.schedule(task2);
    task2Future.get(); // Takes long time
    Thread.sleep(10);
}

task1.Future.get();

Suppose I have a potential of 10000 users using the service at the same time, we can have two kinds of ScheduledExecutorService configuration for my service:

1. A single ScheduledExecutorService for all the users.
2. Create a ScheduledExecutorService for each user.

What I can think about the first method:

Pros:

1. Easy to control the number of threads in the thread pool.
2. Avoid creating new threads for scheduled tasks.

Cons:

1. Always keeping multiple number of threads available could waste computer resources.
2. May cause the hang of the service because of lacking available threads. (For example, set the thread pool size to 10, and then there is a 100 person using the service the same time, then after entering the 1st task and it tries to schedule the 2nd task, then finding out there is no thread available for scheduling the 2nd task)

What I can think about the second method

Pros:

1. Avoiding always keep many threads available when the number of user is small.
2. Can always provide threads for a large number of simultaneously usage.

Cons:

1. Creating new threads creates overheads.
2. Don't know how to control the number of maximum threads for the service. May cause the RAM out of space.

Any ideas about which way is better?

Answer 1

Single ScheduledExecutorService drives many tasks

The entire point of a ScheduledExecutorService is to maintain a collection of tasks to be executed after a certain amount of time elapses.

So given the scenario you describe, you need only a single ScheduledExecutorService object. Submit your 10,000 tasks to that one object. Each task will be executed approximately when its designated delay elapses. Simple, and easy.

Thread pool size

The real issue is deciding how many threads to assign to the ScheduledExecutorService .

Threads, as currently implemented in the OpenJDK project, are mapped directly to host OS threads. This makes them relatively heavyweight in terms of CPU and memory usage. In other words, currently Java threads are “expensive”.

There is no simple easy answer to calculating thread pool size. The optimal number is the least amount of threads that can keep up with the workload without over-burdening the host machine's limited number of cores and limited memory. If you search Stack Overflow, you'll find many discussions on the topic of deciding how many threads to use in a pool.

Project Loom

And keep tabs with the progress of Project Loom and its promise to bring virtual threads to Java. That technology has the potential to radically alter the calculus of deciding thread pool size. Virtual threads will be more efficient with CPU and with memory. In other words, virtual threads will be quite “cheap”, “inexpensive”.

How executor service works

You said:

entering the 1st task and it tries to schedule the 2nd task, then finding out there is no thread available for scheduling the 2nd task

That is not how the scheduled executor service (SES) works.

If a task being currently executed by a SES needs to schedule itself or some other task to later execution, that submitted task is added to the queue maintained internally by the SES. There is no need to have a thread immediately available. Nothing happens immediately except that queue addition. Later, when the added task's specified delay has elapsed, the SES looks for an available thread in its thread-pool to execute that task that was queued a while back in time.

You seem to feel a need to manage the time of each task's execution on certain threads. But that is the job of the scheduled executor service. The SES tracks the tasks submitted for execution, notices when their specified delay elapses, and schedules their execution on a thread from its managed pool of threads. You don't need to manage any of that. Your only challenge is to assign an appropriate number of threads to the pool.

Multiple executor services

You commented:

why don't use multiple ScheduledExecutorService instances

Because in your scenario, there is no benefit. Your Question implies that you have many tasks all similar with none being prioritized. In such a case, just use one executor service. One scheduled executor service with 12 threads will get the same amount of work accomplished as 3 services with 4 threads each.

As for excess threads, they are not a burden. Any thread without a task to execute uses virtually no CPU time. A pool may or may not choose to close some unused threads after a while. But such a policy is up to the implementation of the thread pool of the executor service, and is transparent to us as calling programmers.

If the scenario were different, where some of the tasks block for long periods of time, or where you need to prioritize certain tasks, then you may want to segregate those into a separate executor service.

In today's Java (before Project Loom with virtual threads), when code in a thread blocks, that thread sits there doing nothing but waiting to unblock. Blocking means your code is performing an operation that awaits a response. For example, making.network calls to a socket or web service blocks, writing to storage blocks, and accessing an external database blocks. Ideally, you would not write code that blocks for long periods of time. But sometimes you must.

In such a case where some tasks run long, or conversely you have some tasks that must be prioritized for fast execution, then yes, use multiple executor services.

For example, say you have a 16-core machine with not much else running except your Java app. You might have one executor service with a thread pool size of 4 maximum for long-running tasks, one executor service with a thread pool with a size of 7 maximum for many run-of-the-mill tasks, and a third executor service with a thread pool maximum size of 2 for very few tasks that run short but must run quickly. (The numbers here are arbitrary examples, not a recommendation.)

Other approaches

As commented , there are other frameworks for managing concurrency. The ScheduledExecutorService discussed here is general purpose.

For example, Swing , JavaFX , Spring , and Jakarta EE each have their own concurrency management. Consider using those where approriate to your particular project.