Reading OCR text from a dot matrix display image with Python and OpenCV

Question

I am looking to extract some text from a meter display using Python 3 and OpenCV. I have had some success with a lot of help from SO on the code below, and I can print text from a basic and 'tidy' image file. However when I look to extract from the attached dot matrix image the script is unable to pick out any text at all.

Is there a limitation to extracting in this kind of dot matrix text?

Here's what I am working with:

import cv2
import numpy as np
import pytesseract
from PIL import Image
from cv2 import boundingRect, countNonZero, cvtColor, drawContours, findContours, getStructuringElement, \
    imread, morphologyEx, pyrDown, rectangle, threshold

img = imread('test.JPG')
# down sample and use it for processing
adjusted = pyrDown(img)
# gray-scale image
img_gray = cvtColor(adjusted, cv2.COLOR_BGR2GRAY)

# morph gradient
morph_kernel = getStructuringElement(cv2.MORPH_ELLIPSE, (3, 3))
grad = morphologyEx(img_gray, cv2.MORPH_GRADIENT, morph_kernel)

# change to binary and morph
_, bw = threshold(src=grad, thresh=0, maxval=255, type=cv2.THRESH_BINARY+cv2.THRESH_OTSU)
morph_kernel = getStructuringElement(cv2.MORPH_RECT, (9, 1))
connected = morphologyEx(bw, cv2.MORPH_CLOSE, morph_kernel)
applyMask = np.zeros(bw.shape, np.uint8)

# get contours
im2, contour, hierarchy = findContours(connected, cv2.RETR_CCOMP, cv2.CHAIN_APPROX_SIMPLE)
# filter contours
for index in range(0, len(hierarchy[0])):
    rect = x, y, rectangle_width, rectangle_height = boundingRect(contour[index])

    # draw contour
    mask = drawContours(applyMask, contour, index, (255, 255, 2555), cv2.FILLED)

    # find non-zero pixels ratio
    r = float(countNonZero(applyMask)) / (rectangle_width * rectangle_height)
    if r > 0.5 and rectangle_height > 8 and rectangle_width > 8:
        rec_img = rectangle(adjusted, (x, y+rectangle_height), (x+rectangle_width, y), (0, 255, 0), 3)

        text = pytesseract.image_to_string(Image.fromarray(rec_img))
        print(text)

And here's the image I am trying to extract from:

Answer 1

In my experience, pytesseract doesn't work well in real word. Block Binary Pixel Sum could be better in your case.

Answer 2

This is the best solution I could find. Try blurring the image and then use OpenCV to pull the text. I was able to successfully pull out "Meter Input" but you may need another method to pull out the numbers.

import PIL
from PIL import Image
import PIL.ImageOps
import cv2 as cv
import pytesseract
from PIL import ImageFilter
from pytesseract import image_to_string
pytesseract.pytesseract.tesseract_cmd = r'C:\Program Files\Tesseract-OCR\tesseract.exe'

#open the image and check it out
image = Image.open('dotMatrixScreen.jpg')
display(image)

cropped_img = image.crop((230, 350, 1000, 570))
display(cropped_img)

full_pannel_blur = cropped_img.filter(ImageFilter.BoxBlur(radius=4))    
display(full_pannel_blur)

full_pannel_blur_grey = full_pannel_blur.convert("L")
full_pannel_blur_bw = full_pannel_blur_grey.point(lambda x: 0 if x<122 else 255, '1')
full_pannel_blur_bw.save("full_pannel_blur_bw.png")
display(full_pannel_blur_bw)

full_pannel_bw = full_pannel_blur_bw.convert("L")
full_pannel_bw.save("full_pannel_bw.png")

full_pannel_bw_cv = cv.imread("full_pannel_bw.png", 0)
display(Image.fromarray(full_pannel_bw_cv))

text = image_to_string(full_pannel_bw_cv)
print("Pytesseract got: ", '"' + text + '"')

Out: Pytesseract got: "Meter Index hes. S46m3"