Find Black Squares/Rectangles in Image with C#

Question

I want to find small black squares/rectangles in scanned sheet to:

1. deskewing image if needed.
2. remove white page margin.

Input image: sample image 1 http://us.cdn.persiangig.com/preview/ii2jf6/2.jpg

Output image: sample image 2 http://us.cdn.persiangig.com/preview/9ntpnc/1.jpg

My code to find square is:

Bitmap pic =(Bitmap) pictureBox1.Image;

// create filter
AForge.Imaging.Filters.Median Medianfilter = new AForge.Imaging.Filters.Median();
// apply the filter
Medianfilter.ApplyInPlace(pic);

Bitmap grayImage;

if (pic.PixelFormat != PixelFormat.Format16bppGrayScale && pic.PixelFormat != PixelFormat.Format8bppIndexed)
{
    AForge.Imaging.Filters.Grayscale grayscalefilter = new AForge.Imaging.Filters.Grayscale(0.2125, 0.7154, 0.0721);
    grayImage = grayscalefilter.Apply((Bitmap)pictureBox1.Image);
}
else
{
    grayImage = pic;
}

// black & white:
Threshold(ref grayImage, Convert.ToInt32(textBox1.Text), Convert.ToInt32(textBox1.Text));

// invert filter
Invert invertfilter = new Invert();
// apply the filter
invertfilter.ApplyInPlace(grayImage);

// Edge Detector  filter
DifferenceEdgeDetector EdgeDetectorfilter = new DifferenceEdgeDetector();
// apply the filter
EdgeDetectorfilter.ApplyInPlace(grayImage);

// create filter
Dilatation Dilatationfilter = new Dilatation();
// apply the filter
Dilatationfilter.ApplyInPlace(grayImage);

Finding object(square/rectangle) in image:

        // lock image
        BitmapData bitmapData = grayImage.LockBits(new Rectangle(0, 0, grayImage.Width, grayImage.Height),
        ImageLockMode.ReadWrite, grayImage.PixelFormat);

        // step 2 - locating objects
        BlobCounter blobCounter = new BlobCounter();

        blobCounter.FilterBlobs = true;
        blobCounter.MinHeight = 10;          //*-*-*-*-
        blobCounter.MinWidth = 10;
        blobCounter.MaxHeight = 50;
        blobCounter.MaxWidth = 50;

        blobCounter.ProcessImage(bitmapData);
        Blob[] blobs = blobCounter.GetObjectsInformation();
        grayImage.UnlockBits(bitmapData);

        // step 3 - check objects' type and highlight
        SimpleShapeChecker shapeChecker = new SimpleShapeChecker();

        Graphics g = Graphics.FromImage(pic);
        Pen redPen = new Pen(Color.Red, 2);       // quadrilateral
        Pen bluePen = new Pen(Color.Blue, 2);     // triangle

        for (int i = 0, n = blobs.Length; i < n; i++)
        {
            List<IntPoint> edgePoints = blobCounter.GetBlobsEdgePoints(blobs[i]);

            List<IntPoint> corners;

            // is triangle or quadrilateral
            if (shapeChecker.IsQuadrilateral(edgePoints, out corners))
            {
                // get sub-type
                PolygonSubType subType = shapeChecker.CheckPolygonSubType(corners);

                Pen pen;

                if (subType == PolygonSubType.Square)
                {
                    pen = (corners.Count == 4) ? bluePen : redPen;
                    g.DrawPolygon(pen, ToPointsArray(corners));
                }
            }
        }
        redPen.Dispose();
        bluePen.Dispose();
        g.Dispose();
        pictureBox1.Image = pic;

Problem is low accuracy on detecting squares and rectangles!!!

How can I solve this problem?

Answer 1

If you can use OpenCV this is quite easy. Use the Hough transform and find peaks in the output. Those correspond to straight lines in your input.

If you can't use OpenCV, you'll need to implement it yourself. Here's something to get you started.

https://en.wikipedia.org/?title=Hough_transform

EDIT

As Anders Gustafsson points out in the comment below, the Hough Transform is available in AForge for .NET so implementing it yourself isn't a necessity.

http://www.aforgenet.com/framework/features/hough_transformation.html