Can I run background tasks in a ThreadPool?

Question

I have an ExecutorService to execute my tasks concurrently. Most of these tasks are simple actions that require ~300ms to complete each. But a few of these tasks are background processing queues that take in new sub-tasks all the time and execute them in order. These background tasks will remain active as long as there are normal tasks running.

The ThreadPool is generated through one of the Executors ' methods (don't know which yet) with a user-specified Thread count. My fear is that the following situation might happen: There are less threads than there are background queues. At a given moment, all background queues are working, blocking all the threads of the ExecutorService . No normal tasks will thus be started and the program hang forever.

Is there a possibility this might happen and how can I avoid it? I'm thinking of a possibility to interrupt the background tasks to leave the place to the normal ones.

---

The goal is to limit the number of threads in my application because Google said having a lot of threads is bad and having them idle for most of the time is bad too.

There are ~10000 tasks that are going to be submitted in a very short amount of time at the begin of the program execution. About ~50 background task queues are needed and most of the time will be spent waiting for a background job to do.

Answer 1

Don't mix up long running tasks with short running tasks in same ExecutorService .

Use two different ExecutorService instances with right pool size. Even if you set the size as 50 for background threads with long running tasks, performance of the pool is not optimal since number of available cores (2 core, 4 core, 8 core etc.) is not in that number.

I would like to create two separate ExecutorService initialized with Runtime.getRuntime().availableProcessors()/2 ;

Have a look at below posts for more details to effectively utilize available cores:

How to implement simple threading with a fixed number of worker threads

Dynamic Thread Pool

Answer 2

You can have an unlimited number of threads, check out cache thread pool

Creates a thread pool that creates new threads as needed , but will reuse previously constructed threads when they are available. These pools will typically improve the performance of programs that execute many short-lived asynchronous tasks. Calls to execute will reuse previously constructed threads if available. If no existing thread is available, a new thread will be created and added to the pool. Threads that have not been used for sixty seconds are terminated and removed from the cache. Thus, a pool that remains idle for long enough will not consume any resources. Note that pools with similar properties but different details (for example, timeout parameters) may be created using ThreadPoolExecutor constructors.

Another option is create two different pools and reserve one for priority tasks.

Answer 3

The solution is that the background tasks stop instead of being idle when there is no work and get restarted if there are enough tasks again.

public class BackgroundQueue implements Runnable {

    private final ExecutorService service;
    private final Queue<Runnable> tasks = new ConcurrentLinkedQueue<>();
    private final AtomicBoolean running = new AtomicBoolean(false);
    private Future<?> future;

    public BackgroundQueue(ExecutorService service) {
        this.service = Objects.requireNonNull(service);
        // Create a Future that immediately returns null
        FutureTask f = new FutureTask<>(() -> null);
        f.run();
        future = f;
    }

    public void awaitQueueTermination() throws InterruptedException, ExecutionException {
        do {
            future.get();
        } while (!tasks.isEmpty() || running.get());
    }

    public synchronized void submit(Runnable task) {
        tasks.add(task);
        if (running.compareAndSet(false, true))
            future = service.submit(this);
    }

    @Override
    public void run() {
        while (!running.compareAndSet(tasks.isEmpty(), false)) {
            tasks.remove().run();
        }
    }
}