How many cores do I need to use when I want to benchmark the performance of my compiler?

Question

I reorder the compiler optimization.

And I want to compare the performance of the output with gcc O3.

I have a test-suite.

How many cores do I need to use for benchmark?

I'm sure that the executable files of them are different.

And I use one single core to measure the run time of them, the time is similarly same.

But I don't limit the number of cores to measure the run time, the executable from my compiler is faster than gcc O3.

How can I determine which compiler is better?

Answer 1

---

Question

How many cores do I need to use when I want to benchmark the performance of my compiler?

---

Well, the more the merrier. Single-core as you mentioned is definitely not recommended. Since you have mentioned gcc, you have to look into GCC benchmarks .

However, in the context of aforementioned "the more the merrier" beware of "law of diminishing return" as rightly put by this answer below:

In the benchmark wars the individual manufacturers will will throw as many cores/processors/CPUs at the problem as they can be effective with. But there's always (except in some very weird circumstances) a "law of diminishing return" -- the second core will only add 60-80%, the third core less than that, etc. (And this assumes a problem that is sufficiently multi-threaded to actually make use of the added cores.) So you can't look at a given benchmark and assume that twice as many cores will provide twice the performance. In fact, in some cases you could double the number of cores and actually reduce performance. Achieving good performance in a highly multi-threaded application is somewhere between an art and black magic.