在具有上下文切换的线程池执行器中执行连续运行的线程/任务

Question

I want to run n tasks continuously, however those tasks are memory intensive I want only x of them to be active at a time. But ultimately all those n tasks should run by context switching between them.

In short, I want another implementation of FixedThreadPool , where extra tasks should also run with context switching.

Do we have Thread Pool variant which achieves same? Or any other way to implement it?

UPDATE: After reading a bit and reading answer below, decided to "divide and conquer" ie breaking continuously running task into units of small short lived tasks and submitting those again and again to FixedThreadPool

Answer 1

One could write a paper on the subject but let us keep it simple and concise.

In short, I want another implementation of FixedThreadPoolSize, where extra tasks should also run with context switching.

To achieve that, one would need a Thread Pool that allows to explicitly perform affinity between a Thread and a core. And (as far I know) such a Thread Pool is not officially provided with Java. Which makes sense, since one of the goals of abstractions such as Thread Pools (in Java) is to increase the level of abstraction, even to the point of abstracting concepts such as a thread ( ie, Executor). Therefore, it does not come as a surprise that such a low-level feature (as mapping threads to cores) is not provided out-of-the-box.

Do we have Thread Pool variant which achieves same? Or any other way to implement it?

Unless you are running your code in a Non-Uniform Memory Access (NUMA) architecture, I don't see the benefits of such a low level feature in the context of your program.

My use case is I have to run n tasks continuously. But as those tasks are memory intensive I want only x of them to be active at a time. But ultimately all those n tasks should run by context switching between them.

If you run n tasks and n Threads, and the hardware where the code is running has c cores, where n >> c , then inevitably the SO will map multiple threads to the same core. Hence, you will have your context switching for free .

Finally, before actually opting to run more Threads than cores, profile your code, accordingly. For instance, running the code with the same number threads as cores, then with doubling the number of threads until it stops scaling. It might so happen that your code does even scale with more threads than cores.