OpenCV Finding square center c++

Question

To begin, I am a complete novice in OpenCV and am beginner/reasonable in c++ code. But OpenCV is new to me and I try to learn by doing projects and stuff.

Now for my new project I am trying to find the centre of square in a picture . In my case there is only 1 square in picture. I would like to build further upon the square.cpp example of OpenCV.

For my project there are 2 things I need some help with,

1: The edge of the window is detected as a square, I do not want this. Example

2: How could I get the centre of 1 square from the squares array?

This is the code from the example "square.cpp"

// The "Square Detector" program.
// It loads several images sequentially and tries to find squares in
// each image

#include "opencv2/core.hpp"
#include "opencv2/imgproc.hpp"
#include "opencv2/imgcodecs.hpp"
#include "opencv2/highgui.hpp"

#include <iostream>

using namespace cv;
using namespace std;

static void help(const char* programName)
{
    cout <<
        "\nA program using pyramid scaling, Canny, contours and contour simplification\n"
        "to find squares in a list of images (pic1-6.png)\n"
        "Returns sequence of squares detected on the image.\n"
        "Call:\n"
        "./" << programName << " [file_name (optional)]\n"
        "Using OpenCV version " << CV_VERSION << "\n" << endl;
}


int thresh = 50, N = 11;
const char* wndname = "Square Detection Demo";

// helper function:
// finds a cosine of angle between vectors
// from pt0->pt1 and from pt0->pt2
static double angle(Point pt1, Point pt2, Point pt0)
{
    double dx1 = pt1.x - pt0.x;
    double dy1 = pt1.y - pt0.y;
    double dx2 = pt2.x - pt0.x;
    double dy2 = pt2.y - pt0.y;
    return (dx1 * dx2 + dy1 * dy2) / sqrt((dx1 * dx1 + dy1 * dy1) * (dx2 * dx2 + dy2 * dy2) + 1e-10);
}

// returns sequence of squares detected on the image.
static void findSquares(const Mat& image, vector<vector<Point> >& squares)
{
    squares.clear();

    Mat pyr, timg, gray0(image.size(), CV_8U), gray;

    // down-scale and upscale the image to filter out the noise
    pyrDown(image, pyr, Size(image.cols / 2, image.rows / 2));
    pyrUp(pyr, timg, image.size());
    vector<vector<Point> > contours;

    // find squares in every color plane of the image
    for (int c = 0; c < 3; c++)
    {
        int ch[] = { c, 0 };
        mixChannels(&timg, 1, &gray0, 1, ch, 1);

        // try several threshold levels
        for (int l = 0; l < N; l++)
        {
            // hack: use Canny instead of zero threshold level.
            // Canny helps to catch squares with gradient shading
            if (l == 0)
            {
                // apply Canny. Take the upper threshold from slider
                // and set the lower to 0 (which forces edges merging)
                Canny(gray0, gray, 0, thresh, 5);
                // dilate canny output to remove potential
                // holes between edge segments
                dilate(gray, gray, Mat(), Point(-1, -1));
            }
            else
            {
                // apply threshold if l!=0:
                //     tgray(x,y) = gray(x,y) < (l+1)*255/N ? 255 : 0
                gray = gray0 >= (l + 1) * 255 / N;
            }

            // find contours and store them all as a list
            findContours(gray, contours, RETR_LIST, CHAIN_APPROX_SIMPLE);

            vector<Point> approx;

            // test each contour
            for (size_t i = 0; i < contours.size(); i++)
            {
                // approximate contour with accuracy proportional
                // to the contour perimeter
                approxPolyDP(contours[i], approx, arcLength(contours[i], true) * 0.02, true);

                // square contours should have 4 vertices after approximation
                // relatively large area (to filter out noisy contours)
                // and be convex.
                // Note: absolute value of an area is used because
                // area may be positive or negative - in accordance with the
                // contour orientation
                if (approx.size() == 4 &&
                    fabs(contourArea(approx)) > 1000 &&
                    isContourConvex(approx))
                {
                    double maxCosine = 0;

                    for (int j = 2; j < 5; j++)
                    {
                        // find the maximum cosine of the angle between joint edges
                        double cosine = fabs(angle(approx[j % 4], approx[j - 2], approx[j - 1]));
                        maxCosine = MAX(maxCosine, cosine);
                    }

                    // if cosines of all angles are small
                    // (all angles are ~90 degree) then write quandrange
                    // vertices to resultant sequence
                    if (maxCosine < 0.3)
                        squares.push_back(approx);
                }
            }
        }
    }
}

int main(int argc, char** argv)
{
    static const char* names[] = { "testimg.jpg", 0 };
    help(argv[0]);

    if (argc > 1)
    {
        names[0] = argv[1];
        names[1] = "0";
    }

    for (int i = 0; names[i] != 0; i++)
    {
        string filename = samples::findFile(names[i]);
        Mat image = imread(filename, IMREAD_COLOR);
        if (image.empty())
        {
            cout << "Couldn't load " << filename << endl;
            continue;
        }

        vector<vector<Point> > squares;
        findSquares(image, squares);

        polylines(image, squares, true, Scalar(0, 0, 255), 3, LINE_AA);
        imshow(wndname, image);

        int c = waitKey();
        if (c == 27)
            break;
    }

    return 0;
}

I would like some help to start off. How could I get some information from 1 of the squares out of the array called "squares" (I am having a difficult time understand what exactly is in the array as well; is it an array of points?)

If something is not clear please let me know and I will try to re-explain.

Thank you in advance

Answer 1

Firstly, you are talking about squares but you are actually detecting rectangles. I provided a shorter code to be able to better answer your questions.

I read the image, apply a Canny filter for binarization and detect all contours. After that I iterate through the contours and find the ones which can be approximated by exactly four points and are convex:

int main(int argc, char** argv)
{
    // Reading the images 
    cv::Mat img = cv::imread("squares_image.jpg", cv::IMREAD_GRAYSCALE);
    cv::Mat edge_img;
    std::vector <std::vector<cv::Point>> contours;

    // Convert the image into a binary image using Canny filter - threshold could be automatically determined using OTSU method
    cv::Canny(img, edge_img, 30, 100);

    // Find all contours in the Canny image
    findContours(edge_img, contours, cv::RETR_LIST, cv::CHAIN_APPROX_SIMPLE);

    // Iterate through the contours and test if contours are square
    std::vector<std::vector<cv::Point>> all_rectangles;
    std::vector<cv::Point> single_rectangle;
    for (size_t i = 0; i < contours.size(); i++)
    {

        // 1. Contours should be approximateable as a polygon
        approxPolyDP(contours[i], single_rectangle, arcLength(contours[i], true) * 0.01, true);

        // 2. Contours should have exactly 4 vertices and be convex
        if (single_rectangle.size() == 4 && cv::isContourConvex(single_rectangle))
        {
            // 3. Determine if the polygon is really a square/rectangle using its properties (parallelity, angles etc.)
            // Not necessary for the provided image

            // Push the four points into your vector of squares (could be also std::vector<cv::Rect>)
            all_rectangles.push_back(single_rectangle);
        }
    }

    for (size_t num_contour = 0; num_contour < all_rectangles.size(); ++num_contour) {
        cv::drawContours(img, all_rectangles, num_contour, cv::Scalar::all(-1));
    }

    cv::imshow("Detected rectangles", img);
    cv::waitKey(0); 

    return 0;
    
}

1: The edge of the window is detected as a square, I do not want this.

There are several options depending on your applications. You can filter the outer boundary already using the Canny thresholds, using a different contour retrieval method for finding contours in findContours or by filtering single_rectangle using the area of the found contour (eg cv::contourArea(single_rectangle) < 1000 ).

2: How could I get the centre of 1 square from the squares array?

Since the code is already detecting the four corner points you could eg find the intersection of the diagonals. If you know that there are only rectangles in your image you could also try to detect all centroids of the detected contours using the Hu moments.

I am having a difficult time understand what exactly is in the array as well; is it an array of points?

One contour in OpenCV is always represented as a vector of single points. If you are adding multiple contours you are using a vector of vector of points. In the example you provided squares is a vector of a vector of exactly 4 points. You could also use a vector of cv::Rect in this case.