How to detect and extract signature from an image with OpenCV?

Question

I am importing the attached image. After importing the image, I want to remove horizontal lines, detect the signature and then extract it, create rectangle around signature, crop the rectangle and save it. I am struggling to identify entire region of a signature as one contour or a group of contours.

I have already tried findcontour and then various ways to detect signature region. Please refer the code below.

Python Script:

imagePath

#read image
image = cv2.imread(imagePath,cv2.COLOR_BGR2RGB)

#Convert to greyscale
gray = cv2.cvtColor(image,cv2.COLOR_BGR2GRAY) # grayscale

#Apply threshold
ret,thresh1 = cv2.threshold(gray, 0, 255,cv2.THRESH_OTSU|cv2.THRESH_BINARY_INV)

plt.imshow(thresh1,cmap = 'gray')


#preprocessing
rect_kernel = cv2.getStructuringElement(cv2.MORPH_RECT, (15,15))
dilation = cv2.dilate(thresh1, rect_kernel, iterations = 1)
plt.imshow(dilation,cmap = 'gray')


#Detect contours
contours, hierarchy = cv2.findContours(dilation, cv2.RETR_EXTERNAL, cv2.CHAIN_APPROX_NONE)
contours[0]

height, width, _ = image.shape
min_x, min_y = width, height
max_x = max_y = 0   
for contour, hier in zip(contours, hierarchy):
    (x,y,w,h) = cv2.boundingRect(contour)
    min_x, max_x = min(x, min_x), max(x+w, max_x)
    min_y, max_y = min(y, min_y), max(y+h, max_y)
    if w > 80 and h > 80:
        cv2.rectangle(frame, (x,y), (x+w,y+h), (255, 0, 0), 2)

if max_x - min_x > 0 and max_y - min_y > 0:
    fin=cv2.rectangle(image, (min_x, min_y), (max_x, max_y), (255, 0, 0), 2)

 

plt.imshow(fin)



final=cv2.drawContours(image, contours,-1,(0,0,255),6)

plt.imshow(final,cmap = 'gray')

Final objective is to create rectangle around entire signature

Trying to generalize on the other image:

Answer 1

Instead of removing the horizontal lines, it may be easier to perform HSV color thresholding. The idea is to isolate the signature onto a mask and then extract it. We convert the image to HSV format then use a lower/upper color threshold to generate a mask

lower = np.array([90, 38, 0])
upper = np.array([145, 255, 255])
mask = cv2.inRange(image, lower, upper)

Mask

To detect the signature, we can get the combined bounding box for all of the contours with np.concatenate() then use cv2.boundingRect() to obtain the coordinates

Now that we have the bounding box coordinates, we can use Numpy slicing to crop and extract the ROI

import numpy as np
import cv2

# Load image and HSV color threshold
image = cv2.imread('1.jpg')
hsv = cv2.cvtColor(image, cv2.COLOR_BGR2HSV)
lower = np.array([90, 38, 0])
upper = np.array([145, 255, 255])
mask = cv2.inRange(hsv, lower, upper)
result = cv2.bitwise_and(image, image, mask=mask)
result[mask==0] = (255, 255, 255)

# Find contours on extracted mask, combine boxes, and extract ROI
cnts = cv2.findContours(mask, cv2.RETR_EXTERNAL, cv2.CHAIN_APPROX_SIMPLE)
cnts = cnts[0] if len(cnts) == 2 else cnts[1]
cnts = np.concatenate(cnts)
x,y,w,h = cv2.boundingRect(cnts)
cv2.rectangle(image, (x, y), (x + w, y + h), (36,255,12), 2)
ROI = result[y:y+h, x:x+w]

cv2.imshow('result', result)
cv2.imshow('mask', mask)
cv2.imshow('image', image)
cv2.imshow('ROI', ROI)
cv2.waitKey()

Note: The lower/upper color ranges were obtained from choosing the correct upper and lower HSV boundaries for color detection with cv::inRange (OpenCV)