Why does ThreadPoolExecutor max worker count default to 5 times the number of processes?

Question

In the ThreadPoolExecutor documentation it says:

Changed in version 3.5: If max_workers is None or not given, it will default to the number of processors on the machine, multiplied by 5, assuming that ThreadPoolExecutor is often used to overlap I/O instead of CPU work and the number of workers should be higher than the number of workers for ProcessPoolExecutor .

Is there any reason why ThreadPoolExecutor uses a factor of 5, and is it important to use it for other threading applications in python?

Answer 1

It's probably just been chosen as a reasonable default with the provisions given, ie that those threads will mostly be waiting for IO, not saturating CPU cores.

It might or might not be suitable for your application.

Answer 2

Is there any reason why ThreadPoolExecutor uses a factor of 5, and is it important to use it for other threading applications in python?

1. Most likely because it seems to be a number that works well enough for typical applications. There is no theoretical reason to chose that number... or indeed any number for I/O intensive applications.

2. No. In fact, it is probably a better idea to make the pool size a tuning parameter.