Splitting image according to lines

Question

I'm desperately trying to find my way within openCV to detect lines using HoughLines or any other method, I'm starting from a document image and using structural element and erosion to obtain a binary image with lines.

I managed to obtain the following file but can't seem to obtain HoughLines that are following what seems to me (here is probably the issue) as obvious lines. Any idea on how to go forward or should I start from scratch using other methods ?

The ultimate goal is to extract the lines of the documents as separate images and then try some ML algorithm for handwritten text recognition.

Answer 1

I think that Hough Lines should work in your case. Running

lines = cv2.HoughLines(img_thr, 1, np.pi / 180, threshold=800)

where img_thr is your binary image gives quite good result:

The lines can by sorted by y coordinate of left end (for example) and then two consecutive lines will form a rectangle, which can be extracted using cv2.perspectiveTransform .

There are a few problems, which need to be solved to make this procedure more robust:

• Algorithm can return multiple lines for each line in the image, so they need to be deduplicated.
• There may be some false positive lines, so you need some condition to remove them. I think that looking at the slope of the line and distances between consecutive lines should do the work.
• Effect of threshold parameter in cv2.HoughLines highly depends on the image resolution, so you should resize images to some constant size before running this procedure.

Full code:

img_orig = url_to_image('https://i.stack.imgur.com/PXDKG.png') # orignal image
img_thr = url_to_image('https://i.stack.imgur.com/jZChK.png')  # binary image
h, w, _ = img_thr.shape
img_thr = img_thr[:,:,0]

lines = cv2.HoughLines(img_thr, 1, np.pi / 180, threshold=800)
img_copy = img_orig.copy()
points = []
for rho,theta in lines[:, 0]:
    a, b = np.cos(theta), np.sin(theta)
    x0, y0 = a*rho, b*rho
    x1, x2 = 0, w
    y1 = y0 + a*((0-x0) / -b)
    y2 = y0 + a*((w-x0) / -b)
    cv2.line(img_copy,(int(x1),int(y1)),(int(x2),int(y2)),(255,0,0),4)
    points.append([[x1, y1], [x2, y2]])
points = np.array(points)