块和分配管理中的cuda最大数量

Question

i'm writing a CUDA kernel and I have to execute on this device:

name: GeForce GTX 480
CUDA capability: 2.0
Total global mem:  1610285056
Total constant Mem:  65536
Shared mem per mp:  49152
Registers per mp:  32768
Threads in warp:  32
Max threads per block:  1024
Max thread dimensions:  (1024, 1024, 64)
Max grid dimensions:  (65535, 65535, 65535)

The kernel, in minimal form, is:

_global__ void CUDAvegas( ... )
{
devParameters p;
extern __shared__ double shared[];
int width = Ndim * Nbins;
int ltid = p.lId;
while(ltid < 2* Ndim){
shared[ltid+2*width] = ltid;
ltid += p.lOffset; //offset inside a block
}
__syncthreads();
din2Vec<double> lxp(Ndim, Nbins);

__syncthreads();
for(int i=0; i< Ndim; i++){
  for(int j=0; j< Nbins; j++){
    lxp.v[i][j] = shared[i*Nbins+j];
    }
}
}// end kernel

where Ndim=2, Nbins=128, devParameters is a class whose method p.lId is for counting the local thread's id (inside a block), and din2Cec is a class for creating a Vector of dim Ndim*Nbins whit the new command (in it's destructor I've implemented the corresponds delete[]). The nvcc output is:

nvcc -arch=sm_20   --ptxas-options=-v   file.cu -o file.x
ptxas info    : Compiling entry function '_Z9CUDAvegas4LockidiiPdS0_S0_P7sumAccuP17curandStateXORWOWS0_i' for 'sm_20'
ptxas info    : Function properties for  _Z9CUDAvegas4LockidiiPdS0_S0_P7sumAccuP17curandStateXORWOWS0_i
               0 bytes stack frame, 0 bytes spill stores, 0 bytes spill loads
ptxas info    : Used 22 registers, 116 bytes cmem[0], 51200 bytes cmem[2]

the number of threads is compatible with the MultiProcessors limits: max Shared memory, max register per thread and MP and warps per MP. If I launch 64 threads X 30 blocks (Shared Memory per Block is 4128), it's all right, but if use more than 30 block I obtain the error:

cudaCheckError() failed at file.cu:508 : unspecified launch failure
========= Invalid __global__ read of size 8
=========     at 0x000015d0 in CUDAvegas
=========     by thread (0,0,0) in block (1,0,0)
=========     Address 0x200ffb428 is out of bounds

I think that's a problem in allocating single thread's memory, but I don't understand what's my limit per MP and for total blocks... Someone can help me or remind to a right topic?

PS: I know the kernel presented do nothing, but It's just to understand my limit problems.

Answer 1

I think that the error you receive is explanatory. It is pointed out that there is an out-of-bounds global read of a datatype of size 8. The responsible for the out-of-bounds read is thread (0,0,0) in block (1,0,0). I suspect that responsible instruction is lxp.v[i][j] = shared[i*Nbins+j]; in the last nested for loop. Probably you allocate an amount of global memory which is not related to the number of blocks you launch so that, when you launch too many blocks, you receive such an error.