c++ pthread limit number of thread

Question

I tried to use pthread to do some task faster. I have thousands files (in args) to process and i want to create just a small number of thread many times.
Here's my code :

void callThread(){
    int nbt = 0;
    pthread_t *vp = (pthread_t*)malloc(sizeof(pthread_t)*NBTHREAD);
    for(int i=0;i<args.size();i+=NBTHREAD){
        for(int j=0;j<NBTHREAD;j++){
              if(i+j<args.size()){
                   pthread_create(&vp[j],NULL,calcul,&args[i+j]);   
                   nbt++;
              }
        }
        for(int k=0;k<nbt;k++){
              if(pthread_join(vp[k], NULL)){
                   cout<<"ERROR pthread_join()"<<endl;
              } 
        }
   }
}

It returns error, i don't know if it's a good way to solve my problem. All the resources are in args (vector of struct) and are independants.
Thanks for help.

Answer 1

You're better off making a thread pool with as many threads as the number of cores the cpu has. Then feed the tasks to this pool and let it do its job. You should take a look at this blog post right here for a great example of how to go about creating such thread pool.

A couple of tips that are not mentioned in that post:

• Use std::thread::hardware_concurrency() to get the number of cores.
• Figure out a way how to store the tasks, hint: std::packaged_task or something along those lines wrapped in a class so you can track things such as when a task is done, or implement task.join() .
• Also, github with the code of his implementation plus some extra stuff such as std::future support can be found here .

Answer 2

You can use a semaphore to limit the number of parallel threads, here is a pseudo code:

Semaphore S = MAX_THREADS_AT_A_TIME  //  Initial semaphore value
declare handle_array[NUM_ITERS];

for(i=0 to NUM_ITERS)
{
wait-while(S<=0);
Acquire-Semaphore;  //  S--

handle_array[i] = Run-Thread(MyThread);

}

for(i=0 to NUM_ITERS)
{
Join_thread(handle_array[i])
Close_handle(handle_array[i])
}


MyThread()
{
mutex.lock

critical-section

mutex.unlock

release-semaphore // S++
}