Max number threads in pool

Question

I'm new to multithread programming in C++, and am trying to use thread pools in my code. My code is pretty simple.

#include <iostream>
#include <vector>
#include <thread>

const int SIZE = 100000;

void foo() {
    std::cout << "foo" << std::endl;
}

int main() {
    std::vector<std::thread> myThreads;

    for (int i = 0; i < SIZE; i++) {
        myThreads.push_back(std::thread(foo));
    }

    for (auto& myThread : myThreads) {
        myThread.join();
    }

    return 0;
}

When I run this code from Visual Studio 15 on Windows 10, no problem. It works. My issue is when I run it on my Raspberry Pi 3 I get an error that says:

terminate called after throwing an instance of 'std::system_error'
  what(): Resource temporarily unavailable

Now I assume that what's happening is that the Pi's weak CPU simply can't handle such a large quantity of threads at once. When I change the SIZE down to 100 or 200, the code executes fine.

So why is it that this many threads causes the program to fail? Do the threads not wait to be executed or what is the problem?

Answer 1

Threads take up space. They require memory for a control structure and to store context. In addition, they require a system handle in some environments. System handles are limited. You are probably taxing the ability of the OS on your Raspberry Pi.

Answer 2

Normally folks restrict the number of thread to something like std::thread::hardware_concurrency() to limit it to the number of cores you have on your hardware.

You can create more threads, of course. But not all of them will be able to run, and each thread has its own allocated stack frame. -- So at some point you'll run out of virtual memory on your hardware, a Raspberry Pi.

( It is also possible to fit more in by adjusting your stack size... Just be careful. )

Answer 3

There's no sense to create more thread at once than a number of cores that your system can provide, they just would compete for the CPU time and slow down your application.

As already mentioned, you can get it using std::thread::hardware_concurrency() , but not sure about Raspberry, on some platforms it just returns 0.

If you have a lot of jobs (much more than CPU cores) you should use some thread-pool implementation, that put your jobs into the queue and execute not more than N (that is usually std::thread::hardware_concurrency() ) in a time

You can find plenty of simple thread pool implementations on GitHub, for example