opencv: fit a minimum enclosing ellipse in a binary image

Question

I have used OpenCV and the grabcut implementation to generate a binary mask of the foreground. This is represented as an opencv matrix of CV_8UC1 where the pixels belonging to the foreground has the value 255 and the background is zero (ie it is a binary mask). So, an image like the one attached:

I would like to find the minimum enclosing ellipse for this masked image. The examples that I find online seem a bit complex and I could not translate it to my needs. I tried simply using

// result is my OpenCV array of 
cv::RotatedRect e = cv::fitEllipse(result);

OpenCV Error: Assertion failed (points.checkVector(2) >= 0 && 
(points.depth() == CV_32F || points.depth() == CV_32S)) in fitEllipse, 
file /home/luca/Downloads/opencv-2.4.10/modules/imgproc
/src/contours.cpp, line 2019

terminate called after throwing an instance of 'cv::Exception'
what():  /home/luca/Downloads/opencv-2.4.10/modules/imgproc
/src/contours.cpp:2019: error: (-215) points.checkVector(2) >= 0 && 
(points.depth() == CV_32F || points.depth() == CV_32S) in function 
fitEllipse

The error persists even when I convert it to 32 bit signed int with:

cv::Mat r;
result.convertTo(r, CV_32S);
cv::RotatedRect e = cv::fitEllipse(r);

Answer 1

The function fitEllipse takes an array of cv::Point s, not an image. So you want to run your image through findContours first. Note that findContours modifies the image, so you may want to make a copy first.

std::vector< std::vector<cv::Point> > contours;
cv::Mat tmp = result.clone();
cv::findContours(tmp, contours, CV_RETR_EXTERNAL, CV_CHAIN_APPROX_NONE); // edited to change from CV_CHAIN_APPROX_SIMPLE
cv::RotatedRect e = cv::fitEllipse(contours[0]);

The above assumes you have one and only one contour in your image. You probably want to search contours for the biggest contour (using either size or area) in case there is any noise (and verify that you get at least one contour).

Answer 2

If you are not certain that there is an ellipse in the image, you can use another method by calling cv::minAreaRect .

I've written sample code for 3 different ways:

1. call cv::fitEllipse
2. call cv::minAreaRect
3. call cv::fitEllipse on contour only

code is a little messy, but maybe it will be a help anyways

int main()
{
    cv::Mat input = cv::imread("../inputData/fitEllipseMask.jpg");

    cv::Mat gray;
    cv::cvtColor(input,gray,CV_BGR2GRAY);
    cv::Mat mask = gray > 200; // remove jpeg artifacts

    std::vector<cv::Point> pts;

    for(int j=0; j<mask.rows; ++j)
        for(int i=0; i<mask.cols; ++i)
        {
            if(mask.at<unsigned char>(j,i))
            {
                pts.push_back(cv::Point(i,j));
            }
        }

    cv::RotatedRect result1 = cv::fitEllipse(pts);
    cv::ellipse(input, result1, cv::Scalar(0,255,0) ,3  );

    cv::RotatedRect result2 = cv::minAreaRect(pts);
    cv::ellipse(input, result2, cv::Scalar(0,0,255) ,3 );


    // now a third method to fit an ellipse but only the contour of the mask object

    // edges could be extracted with findContours instead which might or might not be better, depending on input images
    cv::Mat magX, absmagx;
    cv::Sobel(mask, magX, CV_32FC1, 1, 0);
    cv::convertScaleAbs( magX, absmagx );

    cv::Mat magY, absmagy;
    cv::Sobel(mask, magY, CV_32FC1, 0, 1);
    cv::convertScaleAbs( magY, absmagy );

    cv::Mat mag = absmagx+absmagy;

    cv::Mat edgeMask = mag > 0;

    cv::imshow("edges",edgeMask);


    std::vector<cv::Point> ptsEdges;

    for(int j=0; j<edgeMask.rows; ++j)
        for(int i=0; i<edgeMask.cols; ++i)
        {
            if(edgeMask.at<unsigned char>(j,i))
            {
                ptsEdges.push_back(cv::Point(i,j));
            }
        }

    cv::RotatedRect result3 = cv::fitEllipse(ptsEdges);
    cv::ellipse(input, result3, cv::Scalar(255,0,0) , 3  );


    cv::namedWindow("input");
    cv::imshow("input", input);
    //cv::imwrite("../outputData/MainBase.png", input);
    cv::waitKey(0);
    return 0;
}

1. green: since we try to fit the ellipse to the whole white region, the best found ellipse is something like the mean ellipse within the region
2. red: not as good as 3 but will give better results if there is no ellipse in the image but another object
3. blue: fitting an ellipse to a real ellipse with some outliers/noise is just the best result ;)

Answer 3

Ok, I figured it out. I needed to convert it to a contour set. The code is taken from online examples.

cv::vector<cv::vector<cv::Point> > contours;
cv::findContours(result, contours, cv::RETR_LIST, cv::CHAIN_APPROX_NONE);
// Assuming always returns one contour as the input is binary
cv::RotatedRect box = cv::fitEllipse(contours[0]);
// Draw the ellipse
cv::ellipse(image, box, cv::Scalar(255,0,0));