image matching in opencv python

Question

I've been working on a project of recognizing a flag shown in the camera using opencv python. I've already tried using surf, color histogram matching, and template matching. But of these 3, it does not always return the correct answer. what i want now, is what would be the best solution to this problem of mine. Example of the template images:

Here is an example of flag shown in camera.

what to use if this is the kind of images that i want to recognize?

Update code in matchTemplate

    flags=["Cambodia.jpg","Laos.jpg","Malaysia.jpg","Myanmar.jpg","Philippines.jpg","Singapore.jpg","Thailand.jpg","Vietnam.jpg","Indonesia.jpg","Brunei.jpg"]   

    while True:
           methods = 'cv2.TM_CCOEFF_NORMED'
           list_of_pics=[]
           for flag in flags:
                   template= cv2.imread(flag,0)
                   img = cv2.imread('philippines2.jpg',0)

    # generate Gaussian pyramid for A
                    G = template.copy()
                    gpA = [G]
                    for i in xrange(6):
                            G = cv2.pyrDown(G)
                            gpA.append(G)


                    n=0
                    for x in gpA:       

                           w, h = x.shape[::-1]


                           method = eval(methods)#

            # Apply template Match
                          res = cv2.matchTemplate(img,x,method)
                          matchVal=res[0][0]
                          picDict={"matchVal":matchVal,"name":flag}
                          list_of_pics.append(picDict)

                           n=n+1
        newlist = sorted(list_of_pics, key=operator.itemgetter('matchVal'),reverse=True) 
#print newlist
        matched_image=newlist[0]['name']
print matched_image
k=cv2.waitKey(10)
if (k==27):
    break
cv2.destroyAllWindows()

Answer 1

I don't think that you can get good results from SURF/SIFT because:

• SURF/SIFT need keypoints to detect the object but in your case, you have to detect flags and most of the flags are mostly uniform and do not provide much keypoints.

• In your webcam frame, you have several things rather than having only flag. Those several things also contribute to get the keypoints.

Solution: i still think that you should use matchTemplate() of opencv which you have already tried but the problem in your version is that you didn't consider the fact that matchTemplate() is not scale and orientation invariant. So, the solution is to use Gaussian pyramid and create the different size (half, one forth, double etc.) of your sample flags. After getting the same flag in 2-5 different size, you should perform the matchTemplate() between every size of flag and the webcam frame.

Strategy:

• Receive the webcam frame

• Load the image of a flag.

• Using Gaussian pyramid, create smaller and bigger images of that flag (you don't need to store them.)

• Perform matchTemplate() between the webcam frame and each size of flag.

• Result = with which so ever image you get the maximum correlation value is the flag present in your webcam.

REMEMBER: matchTemplate is not scale and orientation invariant. so, if you rotate the image or make it larger/smaller in the webcam frame...you won't get the good results.

Answer 2

SURF cannot be applied to the images that have no corners (when gradient is mostly goes in one direction like in a striped flag). Color histogram of the whole object may not work since both of your examples have similar colors. However, if you can apply a histogram to different parts of the image it will work better.

What you need to do is to split your training image on say 4 quadrants and create 4 color histograms. The testing stage will integrate these 4 back projected histograms and check for the right spatial order of responses. Color histogram is quite robust to rotations, scaling and perspective. It changes with illumination so you need to have liberal matching thresholds. Spatial resolution from 4 quadrants will help to ameliorate this situation.

For the future I recommend studying methods in more detail to understand their applicability rather than trying them randomly.