Cuda优化，多处理器，并发内核执行

Question

I have few questions: (I spent quite some time just trying to find the answers)

1. Where can I find information about maximum number of blocks per streaming multiprocessor, on my device? (I know it might be 16 blocks but can not confirm it) I need to read it like myDevice.maxBlocksPerMultiProcessor inside the code.

2. Will default kernel launch (eg <<<blocks, threads>>> on default stream 0) spread computations evenly among all multiprocessors? (Or is it that only one Multiprocessor will do the work).
   I understand that it depends on my grid configuration, and i am not asking about that. Lets just assume that i have "performance friendly" grid (i mean block-threads / maxThreadPerMultiProcessors maximization of multiprocessors occupancy).
   Will it launch on multiple multiprocessors by default?

3. Lets Say: My GPU supports 16 blocks per multiprocessors & 2048 maxThreadPerMultiProcessors. Then i would like to launch my kernel with <<< N*16, 126 >>> to maximize multiprocessors occupancy. Can I improve performance using streams and / or concurrent kernel execution?
   (I do not think so, because i can not get more then 100% multiprocessor occupancy *i knot it sound absurd but my english is not perfect*)

sorry for my bad english!
thank you for your help!

Answer 1

1. Where can I find information about maximum number of blocks per streaming multiprocessor, on my device?

   You can get this information from the programming guide here . You'll want to know the compute capability of your device. You can look that up here . Your device compute capability can also be retrieved programmatically; study the deviceQuery CUDA sample code for an example. If you need max blocks per multiprocessor programmatically, you will need to incorporate a version of the table in the programming guide linked above into your program, then use the compute capability to determine it at runtime.

2. Will default kernel launch (eg <<<blocks, threads>>> on default stream 0) spread computations evenly among all multiprocessors?

   Yes, this is a fundamental part of the CUDA programming model. As long as you have launched enough blocks to place at least one on each SM, the GPU work distributor will distribute blocks as evenly as it can.

3. Yes, a kernel launch of <<<N, 128>>> where N is sufficiently large, should be an enabling factor to achieve maximum occupancy. Occupancy can have various other limiters (eg registers, shared memory usage, etc.), so this does not guarantee anything, but it should allow for maximum occupancy (2048 threads per SM) in your example. Regarding streams (I think you really mean to ask about concurrent kernels) it's generally true that once you have exposed enough parallelism to saturate a particular GPU, exposing more parallelism may not provide any additional benefit. However, it may provide benefit on a future GPU, and furthermore streams allow for things other than just concurrent kernels. Streams allow for overlap of copy and compute, which may be another valuable factor in improving overall performance.

Many of these topics are covered in the programming guide sections 2-5 on the CUDA programming model, hardware implementation, and performance guidelines. The CUDA best practices guide also covers useful related information.