OpenMP 线程内代码执行时间的测量不正确

Question

So I need to measure execution time of some code inside for loop. Originally, I needed to measure several different activities, so I wrote a timer class to help me with that. After that I tried to speed things up by paralleling the for loop using OpenMP. The problem is that when running my code in parallel my time measurements become really different - the values increase approximately up to a factor of 10. So to avoid the possibility of flaw inside the timer class I started to measure execution time of the whole loop iteration, so structurally my code looks something like this:

#pragma omp parallel for num_threads(20)
for(size_t j = 0; j < entries.size(); ++j)
{
    auto t1 = std::chrono::steady_clock::now();
    // do stuff
    auto t2 = std::chrono::steady_clock::now();

    std::cout << "Execution time is " 
              << std::chrono::duration_cast<std::chrono::duration<double>>(t2 - t1).count()
              << std::endl;
}

Here are some examples of difference between measurements in parallel and measurements in single thread:

Single-threaded	Multi-threaded
11.363545868021	94.154685442
4.8963048650184	16.618173163
4.939025568	25.4751074
18.447368772	110.709813843

Even though it is only a couple of examples, this behaviour seems to prevail in all loop iterations. I also tried to use boost's chrono library and thread_clock but got the same result. Do I misunderstand something? What may be the cause of this? Maybe I get cumulative time of all threads?

Inside the for loop, during each iteration I read a different file. Based on this file I create and solve multitude of mixed-integer optimisation models. I solve them with MIP solver, which I set to run in one thread. The instance of the solver is created on each iteration. The only variables that is shared between iteration are constant strings which represents paths to some directories.

My machine has 32 threads (16 cores, 2 threads per core).

Also here are the timings of the whole application in single-threaded mode:

real    23m17.763s
user    21m46.284s
sys     1m28.187s

and in multi-threaded mode:

real    12m47.657s
user    156m20.479s
sys     2m34.311s

Answer 1

A few points here.

What you're measuring corresponds (roughly) to what time returns as the user time--that is total CPU time consumed by all threads. But when we look at the real time reported by time , we see that your multithreaded code is running close to twice as fast as the single threaded code. So it is scaling to some degree--but not very well.

Reading a file in the parallel region may well be part of this. Even at best, the fastest NVMe SSDs can only support reading from a few (eg, around three or four) threads concurrently before you're using the drive's entire available bandwidth (and if you're doing I/O efficiently that may well be closer to 2. If you're using an actual spinning hard drive, it's usually pretty easy for a single thread to saturate the drive's bandwidth. A PCIe 5 SSD should keep up with more threads, but I kind of doubt even it has the bandwidth to feed 20 threads.

Depending on what parts of the standard library you're using, it's pretty easy to have some "invisible" shared variables. For one common example, quite code that uses Monte Carlo methods will frequently have calls to rand() . Even though it looks like a normal function call, rand() will typically end up using a seed variable that's shared between threads, and every call to rand() not only reads but also writes to that shared variable--so the calls to rand() all end up serialized.

You mention your MIP solver running in a single thread, but say there's a separate instance per thread, leaving it unclear whether the MIP solving code is really one thread shared between the 20 other threads, or that you have one MIP solver instance running in each of the 20 threads. I'd guess the latter, but if it's really the former, then it's being a bottleneck wouldn't seem surprising at all.

Without code to look at, it's impossible to get really specific though.