如何在C ++中设置线程数

Question

I have written the following multi-threaded program for multi-threaded sorting using std::sort. In my program grainSize is a parameter. Since grainSize or the number of threads which can spawn is a system dependent feature. Therefore, I am not getting what should be the optimal value to which I should set the grainSize to? I work on Linux?

 int compare(const char*,const char*)
{
   //some complex user defined logic    
}
void multThreadedSort(vector<unsigned>::iterator data, int len, int grainsize)
{
    if(len < grainsize) 
    {
        std::sort(data, data + len, compare);
    }
    else
    {
        auto future = std::async(multThreadedSort, data, len/2, grainsize);

        multThreadedSort(data + len/2, len/2, grainsize); // No need to spawn another thread just to block the calling thread which would do nothing.

        future.wait();

        std::inplace_merge(data, data + len/2, data + len, compare);
    }
}

int main(int argc, char** argv) {

    vector<unsigned> items;
    int grainSize=10;
    multThreadedSort(items.begin(),items.size(),grainSize);
    std::sort(items.begin(),items.end(),CompareSorter(compare));
    return 0;
}

I need to perform multi-threaded sorting. So, that for sorting large vectors I can take advantage of multiple cores present in today's processor. If anyone is aware of an efficient algorithm then please do share.

I dont know why the value returned by multiThreadedSort() is not sorted, do you see some logical error in it, then please let me know about the same

Answer 1

This gives you the optimal number of threads (such as the number of cores):

unsigned int nThreads = std::thread::hardware_concurrency();

As you wrote it, your effective thread number is not equal to grainSize : it will depend on list size, and will potentially be much more than grainSize.

Just replace grainSize by :

unsigned int grainSize= std::max(items.size()/nThreads, 40);

The 40 is arbitrary but is there to avoid starting threads for sorting to few items which will be suboptimal (the time starting the thread will be larger than sorting the few items). It may be optimized by trial-and-error, and is potentially larger than 40.

You have at least a bug there:

multThreadedSort(data + len/2, len/2, grainsize);

If len is odd (for instance 9), you do not include the last item in the sort. Replace by:

multThreadedSort(data + len/2, len-(len/2), grainsize);

Answer 2

Unless you use a compiler with a totally broken implementation (broken is the wrong word, a better match would be... shitty ), several invocations of std::future should already do the job for you, without having to worry.

Note that std::future is something that conceptually runs asynchronously, ie it may spawn another thread to execute concurrently. May, not must, mind you.
This means that it is perfectly "legitimate" for an implementation to simply spawn one thread per future, and it is also legitimate to never spawn any threads at all and simply execute the task inside wait() .
In practice, sane implementations avoid spawning threads on demand and instead use a threadpool where the number of workers is set to something reasonable according to the system the code runs on.

Note that trying to optimize threading with std::thread::hardware_concurrency() does not really help you because the wording of that function is too loose to be useful. It is perfectly allowable for an implementation to return zero, or a more or less arbitrary "best guess", and there is no mechanism for you to detect whether the returned value is a genuine one or a bullshit value.
There also is no way of discriminating hyperthreaded cores, or any such thing as NUMA awareness, or anything the like. Thus, even if you assume that the number is correct, it is still not very meaningful at all.

On a more general note

The problem "What is the correct number of threads" is hard to solve, if there is a good universal answer at all (I believe there is not). A couple of things to consider:

1. Work groups of 10 are certainly way, way too small . Spawning a thread is an immensely expensive thing (yes, contrary to popular belief that's true for Linux, too) and switching or synchronizing threads is expensive as well. Try "ten thousands", not "tens".
2. Hyperthreaded cores only execute while the other core in the same group is stalled, most commonly on memory I/O (or, when spinning, by the explicit execution of an instruction such as eg REP-NOP on Intel). If you do not have a significant number of memory stalls, extra threads running on hyperthreaded cores will only add context switches, but will not run any faster. For something like sorting (which is all about accessing memory!), you're probably good to go as far as that one goes.
3. Memory bandwidth is usually saturated by one, sometimes 2 cores, rarely more (depends on the actual hardware). Throwing 8 or 12 threads at the problem will usually not increase memory bandwidth but will heighten pressure on shared cache levels (such as L3 if present, and often L2 as well) and the system page manager. For the particular case of sorting (very incoherent access, lots of stalls), the opposite may be the case. May, but needs not be.
4. Due to the above, for the general case "number of real cores" or "number of real cores + 1" is often a much better recommendation.
5. Accessing huge amounts of data with poor locality like with your approach will (single-threaded or multi-threaded) result in a lot of cache/TLB misses and possibly even page faults. That may not only undo any gains from thread parallelism, but it may indeed execute 4-5 orders of magnitude slower. Just think about what a page fault costs you. During a single page fault, you could have sorted a million elements.
6. Contrary to the above "real cores plus 1" general rule, for tasks that involve network or disk I/O which may block for a long time, even "twice the number of cores" may as well be the best match. So... there is really no single "correct" rule.

What's the conclusion of the somewhat self-contradicting points above? After you've implemented it, be sure to benchmark whether it really runs faster, because this is by no means guaranteed to be the case. And unluckily, there's no way of knowing with certitude what's best without having measured.

As another thing, consider that sorting is by no means trivial to parallelize. You are already using std::inplace_merge so you seem to be aware that it's not just "split subranges and sort them".

But think about it, what exactly does your approach really do? You are subdividing (recursively descending) up to some depth, then sorting the subranges concurrently, and merging -- which means overwriting. Then you are sorting (recursively ascending) larger ranges and merging them, until the whole range is sorted. Classic fork-join.
That means you touch some part of memory to sort it (in a pattern which is not cache-friendly), then touch it again to merge it. Then you touch it yet again to sort the larger range, and you touch it yet another time to merge that larger range. With any "luck", different threads will be accessing the memory locations at different times, so you'll have false sharing.
Also, if your understanding of "large data" is the same as mine, this means you are overwriting every memory location beween 20 and 30 times, possibly more often. That's a lot of traffic.

So much memory being read and written to repeatedly, over and over again, and the main bottleneck is memory bandwidth . See where I'm going? Fork-join looks like an ingenious thing, and in academics it probably is... but it isn't certain at all that this runs any faster on a real machine (it might quite possibly be many times slower).

Answer 3

Ideally, you cannot assume more than n*2 thread running in your system. n is number of CPU cores.

Modern OS uses concept of Hyperthreading . So, now on one CPU at a time can run 2 threads.

As mentioned in another answer, in C++11 you can get optimal number of threads using std::thread::hardware_concurrency();