低性能–补丁匹配。 GPU上的图像处理（CUDA）

Question

I have a performance problem: CPU and GPU performances are almost the same.

The Problem I Dealing with is PATCH MATCH. I Have 2 Matrices. I want to find where is the maximum similarity between the big matrix and the small one.

The Matrices has Binary values 0/1 (Black and White).

When I am checking a match between a small matrix to a big one with i5 CPU, it takes 30ms (using multithreading).

When I am checking a match between a small matrix to a big one in a Ge-force GT 730, it takes also 33ms.

I would expect that The GPU will work faster in at least 1 magnitude of order. I pretty disappointed from my current results.

I have two matrices:

1) Big - 300000 (300 rows, 1000 columns)

2) Small 50000 (50 rows, 1000 columns)

The comparing process is done by dividing the big matrix into 250 sub matrices and then comparing each one to the small matrix, then find highest similarity.

The Similarity criterion is the sum of corresponding black pixels on both matrices (the small and the sub-big) divided by the sum of black pixels on sub-big.

I did the last task using the following CUDA code:

 __global__ void matCompare_cuda (uint8_t  *D_SUB , uint8_t  *D_SMALL ,  float *D_RSLTS , unsigned int step, int numOfIndentations ,int SUB_size, int SMALL_size)
{
    int  i = 0 , j = 0 , success = 0, sumZero = 0;    
    int tid = threadIdx.x + blockIdx.x * blockDim.x;

    int LoopIndex = ( tid * step );

    if (tid < numOfIndentations)            
    {
        for ( j = 0 ; j < (SMALL_size) ; j++)
            {
                i = j + LoopIndex;
                if ( D_SUB[i] == 0 )
                    {
                        {
                        sumZero++;
                        if ( D_SMALL[j] == 0 )                
                            success++;            
                        }
                    }
            }
        if (  success > 0 && sumZero > 500)
            D_RSLTS[tid] = 100*((float)success / sumZero) ;                 

    }
}

The Kernal launch:

int numOfIndentations = 300-50  //[ (big.row) - (small.row)]

int numBlock = 16;
int threadNumber = numOfIndentations/numBlock;

matCompare_cuda<<< numBlock , threadNumber >>> ( D_SUB , D_SMALL , D_RSLTS , step, numOfIndentations, SUB_size, SMALL_size ); 

The Cpu Code:

 for (i=0; i < (pixelNum) ; i++)
{    
    if (SUB[i]==0)
    {
        sumDots = sumDots +1;
        if (SMALL->Image[i]==0)
        {
            success = success + 1;
        }    
    }
}


if (success>0)
    if (sumDots>500)    
        RSLT=((float)success/sumDots)*100;

Do you see any improvement that can be done in the GPU code?

Answer 1

A few things. Try to avoid the if's if possible. You can write here:

sumZero += (1 - D_SUB[i])
success += (1 - D_SUB[i]) * (1 - D_SMALL[j])

However I don't think you're going to see a huge difference here. I see two reasons.

One is that there's overhead in invoking cuda. The data needs to be copied to the graphic card and back. That eats some of the speedup you get. Not sure how much it is, but since the run-time is so short it could play a role. I hope you didn't time the compilation of the kernel and other one-time things (take them out by running the code in a loop and ignoring the first few iterations).

Second your big matrix is too small and your small matrix is too big. Because the small matrix is so big (1000 columns) I'm guessing it plays really well with the CPU cache lines. If the small matrix were smaller you would have to go to the next line more often which would increase the chances of breaking the cache line. The gpu uses rectangles for caching so it wouldn't be a problem. If the big matrix were to be bigger you would also increase the amount of computation required so the GPU would start to get ahead.