Java Thread Pool task execution queueing

Question

When using a java thread pool through the ThreadPoolExecutor, are all tasks queued before they are executed? Or are they queued only when there are no threads available (the number of running threads is equal or higher than the core pool size)?

Answer 1

Task submitters always send the tasks to the queue (or try to) , as there's no direct connection between them and the pooled threads ( and for a good reason ).

Unbounded queues

Regarding your question, even if a thread is avaliable, the task will first be inserted in the queue, and then pulled by the choosen free thread. If there are no avaliable threads, the submitters will still place the tasks into the queue. In resume, the flow is always Submitter -> Queue -> Thread .

Unbounded queues often define a max size of INTEGER.MAX_VALUE, which is enough in 99% of the time for avoiding the submitter block.

Size limited queues

If there are no avaliable positions in the queue, the submitters could implement different mechanisms, for example:

• Caller-Runs : In which the submitter is the one who runs the task.
• Blocking behaviour : The submitter may try to insert the task the queue (forever or within a timeout, before discarding it).
• Discard policy : The queue could be configured to remove the oldest queued task if a new one tries to be inserted.

---

Regardless of the mechanism used, the flow of the task (if everything works properly) is:

Submitter -> Queue -> Thread .

In worse scenarios, this flow could be:

Submitter --> Submitter (queue full and caller-runs)

Submitter --> Queue --> GarbageCollector (queue full and discarded as it was the oldest task)

Submitter --> GarbageCollector (queue full and discarded after failed retries)

---

In no case the flow would be

Submitter --> Thread

Avoiding a direct connection between submitters and the thread pool gives this system an asynchronous nature, in which the queue represents the "safety window". This way:

• Submitters are not ever blocked ( using unbounded queues ).
• Submitters are only blocked if the queue is full and the mechanism implements the Blocking behaviour ( using bounded queues ).
• Threads don't have to cope with direct synchronous petitions from the Task Submitters.
• Submitters and Threads avoid complex synchronization mechanisms, as they don't have to negotiate directly.
• If the Submitters stop by some error or are deliverately stopped, Threads could still continue working by pulling the pending queued tasks.
• In the same way, if the Threads are stopped, Submitters could still be able to continue working, placing the tasks in the queue for future processing. (using unbounded queues).

With this in mind, it's important to monitor the queue 's contents and its max size, is order to avoid OOM exceptions. This could happen if submitters send too many tasks and/or too fast, while the threads aren't able to follow their rythm. This would involve an ascending lag inside the queue, hence a possible Out of Memory error.