确定已知字体中的图像中是否存在数字（OCR + OpenCV）

Question

With Python 2.7, I'm trying to recognize digits on playing cards . I've figured out the font being used (Herculanum) and created a reference image of the digits. Multiple cards may be in an image at different angles (upside down for example) and will likely overlap . Eventually, color will come into play, but I want to identify numbers first.

Ideally, I would want to know of presence of a number and possibly its location in the image. I've tried a few methods using pytesseract and OpenCV. I'm also in the process of training a TensorFlow neural network, but I think it's overkill for what seems to be a simple problem. I have some code to read in the reference digits, but I'm struggling to tie it back to an example card image.

from imutils import contours
import numpy as np
import imutils
import cv2

ref = cv2.imread('ocr_a_reference.png')
ref = cv2.cvtColor(ref, cv2.COLOR_BGR2GRAY)
ref = cv2.threshold(ref, 10, 255, cv2.THRESH_BINARY_INV)[1]

refCnts = cv2.findContours(ref.copy(), cv2.RETR_EXTERNAL, 
cv2.CHAIN_APPROX_SIMPLE)
refCnts = refCnts[0] if imutils.is_cv2() else refCnts[1]
refCnts = contours.sort_contours(refCnts, method="left-to-right")[0]
digits = {}

for (i, c) in enumerate(refCnts):
    # compute the bounding box for the digit, extract it, and resize
    # it to a fixed size
    (x, y, w, h) = cv2.boundingRect(c)
    roi = ref[y:y + h, x:x + w]
    roi = cv2.resize(roi, (57, 88))

    # update the digits dictionary, mapping the digit name to the ROI
    digits[i] = roi

From a known font, is it possible to identify matching digits in an image? It would almost be like searching for an image within an bigger image for the closest match. The ideal end output would be a list of digits and the count within the image. Is this possible with pytesseract and/or OpenCV? Or is there another library I should be looking at?

Thanks for the advice!

Answer 1

You can use tesseract to iterate over words that it has identified and get the bounding box for each word.However, you will have the following challenge :

• Orientation : While you can configure tesseract to auto detect orientation , your image has multiple orientations for different cards which contain the digits.
• Duplicates : You will have same digits visible twice on some card and only once on others if they are hidden by other cards.

One approach you might be able to use is that you can generate your reference digits for font used in card at multiple rotations for ex: every 10 degress...As a result of this you will have 36 reference images for each character and 360 in all. You can now train tesseract on this model. Also, i notice your card has black color used only for digits and border. You can preprocess your image to remove remaining background colors to improve accuracy..You might have to retry with different angles if you are not getting very good result with 10 degree rotations. For duplicate issue, once you have bounding box for all digits and you know that digits in a given card will have fixed distance from its opposite pair. you can use this information to remove any duplicates.