OpenCV - C ++到Java - 模板匹配

Question

I am trying to detect more than a square (marker) in my image as I ask here Detect Marker Position in 2D image

There is one guy that showed me a solution in C++, here it is:

#include <iostream>

#include <opencv2/highgui/highgui.hpp>
#include <opencv2/imgproc/imgproc.hpp>


//See: http://docs.opencv.org/doc/tutorials/imgproc/histograms/template_matching/template_matching.html
//See: http://answers.opencv.org/question/60382/detect-markers-position-in-2d-images/
int main() {
  cv::Mat img, templateImg, result;
  cv::VideoCapture capture("http://answers.opencv.org/upfiles/14297307634571599.png");
  if(capture.isOpened()) {
    capture >> img;
  } else {
    return -1;
  }

  capture = cv::VideoCapture("http://answers.opencv.org/upfiles/14297308125543022.png");
  if(capture.isOpened()) {
    capture >> templateImg;
  } else {
    return -1;
  }

  /// Reduce the size of the image to display it on my screen
  cv::resize(img, img, cv::Size(), 0.5, 0.5);
  /// Reduce the size of the template image
  /// (first to fit the size used to create the image test, second to fit the size of the reduced image)
  cv::resize(templateImg, templateImg, cv::Size(), 0.25, 0.25);

  cv::Mat img_display;
  img.copyTo(img_display);

  // Create the result matrix
  int result_cols =  img.cols - templateImg.cols + 1;
  int result_rows = img.rows - templateImg.rows + 1;

  result.create(result_rows, result_cols, CV_32FC1);

  /// Do the Matching and Normalize
  cv::matchTemplate(img, templateImg, result, CV_TM_CCORR_NORMED);
  cv::normalize(result, result, 0, 1, cv::NORM_MINMAX, -1, cv::Mat());

  /// Localizing the best match with minMaxLoc
  double minVal; double maxVal; cv::Point minLoc; cv::Point maxLoc;
  cv::Point matchLoc;

  for(;;) {
    cv::minMaxLoc(result, &minVal, &maxVal, &minLoc, &maxLoc, cv::Mat());
    matchLoc = maxLoc;
    std::cout << "Max correlation=" << maxVal << std::endl;
    if(maxVal < 0.8) {
      break;
    }

    /// Show me what you got
    cv::rectangle(img_display, matchLoc, cv::Point(matchLoc.x + templateImg.cols , matchLoc.y + templateImg.rows),
        cv::Scalar::all(0), 2, 8, 0);
    cv::rectangle(result, cv::Point(matchLoc.x - templateImg.cols/2 , matchLoc.y - templateImg.rows/2),
        cv::Point(matchLoc.x + templateImg.cols/2 , matchLoc.y + templateImg.rows/2 ), cv::Scalar::all(0), 2, 8, 0);

    cv::imshow("result", result);
    cv::waitKey(0);

    /// Fill the detected location with a rectangle of zero
    cv::rectangle(result, cv::Point( matchLoc.x - templateImg.cols/2 , matchLoc.y - templateImg.rows/2),
        cv::Point(matchLoc.x + templateImg.cols/2 , matchLoc.y + templateImg.rows/2 ), cv::Scalar::all(0), -1);
  } while (maxVal > 0.9);


  cv::imshow("result", result);
  cv::imshow("img_display", img_display);
  cv::waitKey(0);

  return 0;
}

The for loop is responsible to find more than one marker and detect it, I am trying to adapt it to my java code and I am getting an infinite loop here is my code:

public void run(String inFile, String templateFile, String outFile, int match_method) {
        System.out.println("\nRunning Template Matching");


    Mat img = Highgui.imread(inFile);
    Mat templ = Highgui.imread(templateFile);

    // / Create the result matrix
    int result_cols = img.cols() - templ.cols() + 1;
    int result_rows = img.rows() - templ.rows() + 1;
    Mat result = new Mat(result_rows, result_cols, CvType.CV_32FC1);

    // / Do the Matching and Normalize
    Imgproc.matchTemplate(img, templ, result, match_method);
    Core.normalize(result, result, 0, 1, Core.NORM_MINMAX, -1, new Mat());

    Point matchLoc;
    Point maxLoc;
    Point minLoc;

    MinMaxLocResult mmr;

    boolean iterate = true;
    while(iterate){

    // / Localizing the best match with minMaxLoc
    mmr = Core.minMaxLoc(result);
    matchLoc = mmr.maxLoc;


    if(mmr.maxVal < 0.8)
    {
        iterate = false;
    }



    // / Show me what you got
    Core.rectangle(img, matchLoc, new Point(matchLoc.x + templ.cols(),
            matchLoc.y + templ.rows()), new Scalar(0, 255, 0));

    }

    // Save the visualized detection.
    System.out.println("Writing "+ outFile);
    Highgui.imwrite(outFile, img);

}

I notice that the function minMaxLoc has more arguments in c++ than in java, maybe that's the problem? Why I can't get the same behaviour in java can someone help me?

Thank you so much in advance

Answer 1

As cyriel said, You forgot to fill zeros for the maximum location and hence you got the infinite loop. May be he forgot to explain you that,

   for each iteration
   find the max location
   check if max value is greater than desired threshold
   if true 
   show me what is max
   else
   break // not found anything that matches
   make the existing max to be zero and continue to search for other max// you forgot this and hence infinite loop
   end

For more insight to the problem, you can comment the following lines in the c++ code and run it, you will experience similar problem. (Here cv::Scalar::all(0),-1 args says fill the existing max/found region with 0 and continue)

// Fill the detected location with a rectangle of zero
    cv::rectangle(result, cv::Point(matchLoc.x - templateImg.cols / 2, matchLoc.y - templateImg.rows / 2),
        cv::Point(matchLoc.x + templateImg.cols / 2, matchLoc.y + templateImg.rows / 2), cv::Scalar::all(0), -1);

Hope it helps.

Answer 2

You forgot to draw second rectangle. Original code:

 cv::rectangle(result, cv::Point(matchLoc.x - templateImg.cols/2 , matchLoc.y - templateImg.rows/2),
    cv::Point(matchLoc.x + templateImg.cols/2 , matchLoc.y + templateImg.rows/2 ), cv::Scalar::all(0), 2, 8, 0);

...

/// Fill the detected location with a rectangle of zero
cv::rectangle(result, cv::Point( matchLoc.x - templateImg.cols/2 , matchLoc.y - templateImg.rows/2),
    cv::Point(matchLoc.x + templateImg.cols/2 , matchLoc.y + templateImg.rows/2 ), cv::Scalar::all(0), -1);

however in your code there is only this:

Core.rectangle(img, matchLoc, new Point(matchLoc.x + templ.cols(),
        matchLoc.y + templ.rows()), new Scalar(0, 255, 0));

The second part of original code most likely is crucial - it draws filled, not just a shape like first line.

Btw this line Core.rectangle(img, matchLoc, new Point(matchLoc.x + templ.cols(), matchLoc.y + templ.rows()), new Scalar(0, 255, 0)); is probably wrong as well - 1) You should draw rectangle on result mat, not on img mat. 2) In original code there is Core.rectangle(img, matchLoc, new Point(matchLoc.x + templ.cols(), matchLoc.y + templ.rows()), new Scalar(0, 255, 0)); and in your code new Point(matchLoc.x + templ.cols(), matchLoc.y + templ.rows()) . Are you sure it is fine?

Answer 3

Although it might help, you don't need OpenCV / JavaCV for this relatively easy task.

Basically, you want to find regions in you image that are:

a: all black
b: square
c: a percentage of size of the total image

First, threshold the image so all pixels are either black or white.

My approach would be to find the connected components by scanning the image pixel by pixel until you reach a black pixel. Then 'flood fill' until you have all the black pixels collected. Mark them (color them differently) and store them in some structure (list of points) and repeat until you reached the last pixel of the image.

For each component:

• determine the bounding box (min and max x and y values).
• Calculate the size of the box and determine if it is small / large enough
• Calculate the percentage of matching pixels in that box. If it is close to 100%, you have a square shape.

You can plot the matching bounding boxes back into your original image.