ThreadPoolExecutor 线程数

Question

I am trying to use the ThreadPoolExecutor in Python using the futures backport package. However, the problem is that all threads are executed at the same time so there is no actual pooling take place. More specifically I get 10 threads of that function instead of 5 and then the others. I use the following code do you find something wrong or it is just the backported implementation? Thank you!

with ThreadPoolExecutor(max_workers=5) as executor:
    futures = [executor.submit(f, X, y) for t in range(10)]
    for future in as_completed(futures):
        self.trees.append(future.result())

Answer 1

In the documentation it says that it's typical to have multiple threads per worker:

https://docs.python.org/3/library/concurrent.futures.html

Changed in version 3.5: If max_workers is None or not given, it will default to the number of processors on the machine, multiplied by 5, assuming that ThreadPoolExecutor is often used to overlap I/O instead of CPU work and the number of workers should be higher than the number of workers for ProcessPoolExecutor.

Probably it doesn't default to 5 multiplier because you're using a version other than 3.5+ or for some other internal optimization reason.