Deskew Text with OpenCV and Python (RotatedRect, minAreaRect)

Question

I'm new with OpenCV and I want to deskew an image that have a skewed text. First I read the image in GrayScale and Binarize it, then I try to do this :

import cv2
import numpy as np

img = cv2.imread('m20.jpg',0)
ret,byw = cv2.threshold(img,127,255,cv2.THRESH_BINARY_INV)
_, contours, hierarchy = cv2.findContours(byw.copy(), cv2.RETR_TREE,cv2.CHAIN_APPROX_SIMPLE)
cnt = contours[0]
draw = cv2.cvtColor(byw, cv2.COLOR_GRAY2BGR)
rect = cv2.minAreaRect(cnt)
box = cv2.boxPoints(rect)
box = np.int0(box)
cv2.drawContours(draw, [box], 0, (0, 255, 0), 2)

But doesn't work beacause findContours() expect to receive an image with a body shape. Other way that i try is to translate this code of c++:

// Read image
Mat3b img = imread("path_to_image");

// Binarize image. Text is white, background is black
Mat1b bin;
cvtColor(img, bin, COLOR_BGR2GRAY);

bin = bin < 200;

// Find all white pixels
vector<Point> pts;
findNonZero(bin, pts);

// Get rotated rect of white pixels
RotatedRect box = minAreaRect(pts);
if (box.size.width > box.size.height)
{
    swap(box.size.width, box.size.height);
    box.angle += 90.f;
}

Point2f vertices[4];
box.points(vertices);

for (int i = 0; i < 4; ++i)
{
    line(img, vertices[i], vertices[(i + 1) % 4], Scalar(0, 255, 0));
}

// Rotate the image according to the found angle
Mat1b rotated;
Mat M = getRotationMatrix2D(box.center, box.angle, 1.0);
warpAffine(bin, rotated, M, bin.size());

And i have this:

draw = cv2.cvtColor(byw, cv2.COLOR_GRAY2BGR)
data = np.array(byw)
subzero = np.nonzero(data)
subuno = np.reshape(subzero,(17345,2)) # this is because cv2.minAreaRect() receives a Nx2 numpy
rect = cv2.minAreaRect(subuno)
box = cv2.boxPoints(rect)
box = np.int0(box)
cv2.drawContours(draw,[box],0,(0,255,0),2)

But then again the result is not the expected, the rectangle is not well positioned.

---

Also occurs to me that might try to make the for like in C++ but I don't know how to obtain the vertices from the box = cv2.boxPoints(rect) . Please help!

Answer 1

Maybe you can check this out: http://www.pyimagesearch.com/2014/08/25/4-point-opencv-getperspective-transform-example/

In that link, the author deskews or transforms the entire document (and thus also the contained text), however, it depends on finding the edges of the document, based on the contours found in an image.

He takes it further in this following tutorial: http://www.pyimagesearch.com/2014/09/01/build-kick-ass-mobile-document-scanner-just-5-minutes/

His solutions work because he can adjust the entire document based on the detected position, orientation, and skewness of the documents. Adjusting the position of the document as a whole in effect adjusts everything found inside the document, including the text.

However, I believe what you're asking is you want to deskew text even without detecting any document edges and contours. If this is the case, then I'm supposing that you'll need to provide another basis or standard to base your text deskewing on (ie detect that there are letters in the image, then detect how skewed the letters are based on your standard, then adjust the letters), which may be a non-trivial exercise.