Cuda AtomicAdd不递增

Question

I'm stuck on such a stupid problem I suppose. This is a test kernel just to see the atomicAdd working:

__global__
void pixelcount_kernel(unsigned int * d_count,
                      const size_t numElems)
{ 
int myId = threadIdx.x + blockDim.x * blockIdx.x;
//avoid out of boundary access
if(myId > (numElems-1))
{return;
}

unsigned int inc=1;
atomicAdd(d_count, inc);
//debug code
printf("d_count: %d \n", *d_count);   
}

and this is the memory allocation,initialization and kernel call:

unsigned int* d_count;
checkCudaErrors(cudaMalloc(&d_count, sizeof(unsigned int)));
checkCudaErrors(cudaMemset(d_count, 0, sizeof(unsigned int)));
pixelcount_kernel<<<gridSize, blockSize>>>( d_count, 10);

In the output I don't see any increment from 0 to numElems (10 in this call), but this:

d_count: 10 
d_count: 10 
d_count: 10 
d_count: 10 
d_count: 10 
d_count: 10 
d_count: 10 
d_count: 10 
d_count: 10 
d_count: 10

What's wrong? Thanks Giuseppe

Answer 1

There is nothing wrong with this output. Different threads of a kernel launch execute in parallel, so it is entirely legit that all threads have incremented d_count by the time the first thread reaches printf().

In fact, if you are running the code with a blocksize of at least 10, all 10 threads are part of the same warp and are certain to execute the same instruction at the same time.

If you want to see d_count incrementing (eg if you want to give each participating thread a unique id), use this code instead:

unsigned int my_d_count = atomicAdd(d_count, inc);
printf("d_count before atomic increment: %d \n", my_d_count);