如何防止编译器优化对基准read（）与mmap（）性能的内存访问？

Question

I would like to benchmark read() vs mmap() performance of a C program reading 10GB of data. If I have read or mmap'ed the data to a buffer, what should be done in order to make sure the data was actually read?

At the moment I use the following function after each single read() and after the one mmap() operation to make sure data is actually in memory:

void use_data(void *data, size_t length) {
    volatile int c = 0;
    for (size_t i = 0; i < length; i++) {
        c += *((char *) data + i);
    }
}

However, I feel this might even introduce overhead? Maybe one can even distinguish between read() and mmap():

In the read() case I think no explicit data access is needed, because the read() call will copy the data to a buffer anyway. In the case of mmap() however, I think some kind of summing up/counting need to be performed in order to make the kernel load every page.

Any recommendations?

Answer 1

You don't need to access the volatile variable for each byte you process. Sum all bytes into a local. Then, write the sum into a volatile variable.

In fact you don't need a volatile variable. You can use any opaque sink that the compiler cannot prove as unneeded. Writing the sum to a temp file would be guaranteed to work as well.

Note, that this is not just a hack to make the compiler cooperate. This is guaranteed to touch every byte (because it could influence the result). The result is needed for an external IO. This cannot be optimized away under the standard.

If alignment allows, sum in bigger units such as 32 or 64 bits. Use unsigned types to avoid UB on overflow. You want to be memory/IO bound, not ALU bound. You can create instruction-level parallelism by summing multiple independent streams using multiple local accumulator variables.