Thread safe array for std::thread?

Question

mystruct** = (mystruct**)calloc(10, sizeof(mystruct*);
for(unsignd int i = 0; i < 10; i++)
    mystruct[i] = (mystruct*)calloc(10, sizeof(mystruct);

thread t[10];

for(unsigned int i = 0; i < 10; i++)
    t[i] = thread(new_piece, mystruct[i]);

for(unsigned int i = 0; i < 10; i++)
    t[i].join();

The function new_piece writes data to mystruct[i] . To be more specific, the function changes the values of mystruct[i][0], mystruct[i][1], ..., mystruct[9]

How to make the above operation thread safe?

Answer 1

As mentioned in the comments already, the code seems to be "thread-safe", however it might suffer from "cache-thrashing".

First let me explain what it is, and why this might happen in your code:

What is cache-thrashing:

A "cache-line" is the smallest unit of data which gets fetched into the cache from memory. Note that the size of the cache-line is a hardware property. There's no language construct which would yield this value. Most cache-line sizes on modern CPUs are 64 bytes.

Cache-thrashing occurs on systems with multiple CPUs and a coherent cache when different threads access distinct variables which happen to be layout in memory which share the same cache line. This will lead to excessive cache misses, and thus results in degraded performance.

(See also wiki article false-sharing ) which refers to the usage pattern which causes cache-thrashing)

(See also: Dr.Dobb's article Eliminate False Sharing )

Why might it happen in your code:

Allocation blocks returned from calloc or malloc strive to be as small as possible and are packed tightly together. That is, different allocation blocks may share the same cache-line (see also man 3 calloc , eg FreeBSD man page ).

C++'s new operator won't be different.

Now, that we cannot generally assume that memory blocks returned from calloc or malloc will not share a common cache-line, your code may suffer from cache-thrashing, since your instances of mystruct which are accessed simultaneously from different threads may share a common cache-line.

How to avoid "cache-thrashing":

Basically, you ensure this through properly aligning your data (see wiki article Data Structure Alignment ).

There are many approaches where you can ensure your data ( mystruct ) is properly cache-aligned, for example:

• Use malloc or calloc and round your allocation requests up to the nearest multiple of the cache-line size.

• utilise a posix function: posix_memalign , declared in header <stdlib.h> (see opengroup posix_memalign )

• In C++11, you can use std::aligned_storage (see definition here )

  std::aligned_storage provides for a type which is suitable for use as uninitialized storage where you can store your object.

  For example, defining a cache-line aligned storage which is an array for N instances:

   struct mystruct { ... }; const std::size_t cache_line_size = 64; typename std::aligned_storage<sizeof(mystruct), cache_line_size>::type storage[N]; 

  With that, you now could define a class which wraps an array of N cache-line aligned mystruct s, which also provides convenient accessor functions to modify and retrieve a mystruct value at position i in the underlying array. IMO, this approach would be much preferred over your error prone approach using a loop and calloc for instantiating the storage for your mystruct array.

  See the example here , which - slightly modified - would perfectly fit your needs.