使用 CudaEventElapsedTime 测量 Cuda 内核时间

Question

I've got NVS 5400M and I'm trying to get reliable time measurement results for cuda addition on matrix (instance 1000 x 1000).

__global__ void MatAdd(int** A, int** B, int** C) {
int i = threadIdx.x;
int j = threadIdx.y;
C[i][j] = A[i][j] + B[i][j]; }

And I'm doing measurement like:

int numBlocks = 1;
dim3 threadsPerBlock(1000, 1000);

float time;
cudaEvent_t start, stop;
cudaEventCreate(&start);
cudaEventCreate(&stop);
cudaEventRecord(start, 0);

MatAdd <<<numBlocks, threadsPerBlock>>>(pA, pB, pC);

cudaEventRecord(stop, 0);
cudaEventSynchronize(stop);
cudaEventElapsedTime(&time, start, stop);

cout << setprecision(10) << "GPU Time [ms] " << time << endl;

and the result is: 0.001504000043 ms, which is relatively small. My question is am I doing it right?

Answer 1

Your timing is correct, but your usage of CUDA in general is not.

This is illegal:

dim3 threadsPerBlock(1000, 1000);

CUDA kernels are limited to a maximum of 1024 threads per block, but you are requesting 1000x1000 = 1,000,000 threads per block.

As a result, your kernel is not actually launching:

MatAdd <<<numBlocks, threadsPerBlock>>>(pA, pB, pC);

And so the measured time is quite short.

You are advised to use proper cuda error checking and run your tests with cuda-memcheck to make sure there are no reported runtime errors (my guess is right now you are not even aware of the errors being reported from your code - you have to check for them.)

Since you haven't shown a complete code, I'm not going to try to identify all other issues that may be present, but your kernel code would have to be re-factored in order to handle a 1000x1000 array properly, and passing double-pointer (eg int** A ) parameters to kernels is considerably more difficult than a single pointer or "flat" array.