为什么在CUDA中启动多个32个线程？

Question

我参加了CUDA并行编程课程，我已经看到很多CUDA线程配置的例子，其中通常将所需的线程数量向上舍入为32的最接近的倍数。我理解线程被分组为warp，如果你启动1000个线程，GPU无论如何都会将其四舍五入，那么为什么要明确呢？

Answer 1

The advice is generally given in the context of situations where you might conceivably choose various threadblock sizes to solve the same problem.

Let's take vector add as an example. Suppose my vector is of length 100000. I might choose to do this by launching 100 blocks of 1000 threads each. In this case, each block will have 1000 active threads, and 24 inactive threads. My average utilization of thread resources is 1000/1024 = 97.6%.

Now, what if I chose blocks of size 1024? Now I only need to launch 98 blocks. The first 97 of these blocks are fully utilized in terms of thread utilization - every thread is doing some thing useful. The 98th block only has 672 (out of 1024) threads that are doing something useful. The others are explicitly inactive because of a thread check ( if (idx < N) ) or other construct in the kernel code. So I have 352 inactive threads in that one block. But my overall average utilization is 100000/100352 = 99.6%

So in the above scenario, it's better to choose "full" threadblocks, evenly divisible by 32.

If you are doing vector add on a vector of length 1000, and you intend to do it in a single threadblock, (both may be bad ideas), then it does not matter whether you choose 1000 or 1024 for your threadblock size.