Threading by partitioning range

Question

I have a vector of threads that I'm executing; but instead of firing off 1,000 threads all at once (resulting in tons of context-switching), I was hoping to do 8 at a time and wait for completion of those 8 before continuing on; however my code only does the first 8 then exits the loop... unsure why. Any help is appreciated!

Note: Simplified code below

    for (int _max_orders : max_orders)
    {
        ThreadVector.emplace_back([=]() {execute_with_params(_max_orders); });
        if (++thread_count == 8)
        {
            for (auto& t : ThreadVector)
            {
                t.join();
            }
            thread_count = 0;
        }
    }

Answer 1

The simple act of creating a thread has enough overhead that creating and destroying 1000 of them will typically have a noticeable impact on performance.

Instead of creating 1000 threads, then executing only 8 (or whatever number) of them at once, I'd consider creating a pool of 8 threads. Each thread would sit in a loop getting the next task to execute. A simple (but working) example would be something like this:

#include <cstdio>
#include <thread>
#include <atomic>
#include <vector>

void execute_with_params(int param) {
    std::printf("%d\t", param);
}

class next_task {
    std::atomic<int> task{0};
    std::atomic<int> max;

public:
    next_task(int max) : max(max) {}

    int operator()() { 
        int next = task++; 
        return next < max ? next : -1;
    }
};

template <class Task>
class executor {
    Task &next_task;
public:
    executor(Task &t) : next_task(t) {}

    void operator()() {
        int task;
        while (-1 != (task = next_task()))
            execute_with_params(task);        
    }    
};

int main() {
    int num_threads = 8;

    next_task tasks(1000);

    std::vector<std::thread> threads;

    for (int i=0; i<num_threads; i++)
        threads.emplace_back(std::thread(executor<next_task>(tasks)));

    for (auto &t : threads)
        t.join();
}