Extract an arbitrary rectangular patch from an image given 4 point coordinates

Question

Given the coordinates of four arbitrary points in an image (which are guaranteed to form a rectangle), I want to extract the patch that they represent and get a vectorized (flat) representation of the same. How can I do this?

I saw the answer to this question and using it I am able to reach to the patch that I require. For example, given the image coordinates of the 4 corners of the green rectangle in this image:

I am able to get to the patch and get something like:

using the following code:

p1 = (334,128)
p2 = (438,189)
p3 = (396,261)
p4 = (292,200)
pts = np.array([p1, p2, p3, p4])

mask = np.zeros((img.shape[0], img.shape[1]))

cv2.fillConvexPoly(mask, pts, 1)
mask = mask.astype(np.bool)

out = np.zeros_like(img)
out[mask] = img[mask]
patch = img[mask]

cv2.imwrite(img_name, out)

However, the problem is that the patch variable that I obtain is simply an array of all pixels of the image that belong to the patch, when the image is read as a matrix in row-major order.

What I want is that patch variable should contain the pixels in the order they can form a genuine image so that I can perform operations on it. Is there an opencv function that I should be aware of that would help me in doing this?

Thanks!

Answer 1

This is how you can implement this:

Code:

    # create a subimage with the outer limits of the points
    subimg = out[128:261,292:438]

    # calculate the angle between the 2 'lowest' points, the 'bottom' line
    myradians = math.atan2(p3[0]-p4[0], p3[1]-p4[1])
    # convert to degrees 
    mydegrees = 90-math.degrees(myradians)

    # create rotationmatrix
    h,w = subimg.shape[:2]
    center = (h/2,w/2)
    M = cv2.getRotationMatrix2D(center, mydegrees, 1)
    # rotate subimage
    rotatedImg = cv2.warpAffine(subimg, M, (h, w))

Result:

Next, the black areas in the image can be easily cropped by removing all rows/columns that are 100% black.
Final result:

Code:

    # converto image to grayscale
    img = cv2.cvtColor(rotatedImg, cv2.COLOR_BGR2GRAY)

    # sum each row and each volumn of the image
    sumOfCols = np.sum(img, axis=0)
    sumOfRows = np.sum(img, axis=1)

    # Find the first and last row / column that has a sum value greater than zero, 
    # which means its not all black. Store the found values in variables
    for i in range(len(sumOfCols)):
            if sumOfCols[i] > 0:
                    x1 = i
                    print('First col: ' + str(i))
                    break

    for i in range(len(sumOfCols)-1,-1,-1):
            if sumOfCols[i] > 0:
                    x2 = i
                    print('Last col: ' + str(i))
                    break

    for i in range(len(sumOfRows)):
            if sumOfRows[i] > 0:
                    y1 = i
                    print('First row: ' + str(i))
                    break

    for i in range(len(sumOfRows)-1,-1,-1):
            if sumOfRows[i] > 0:
                    y2 = i
                    print('Last row: ' + str(i))
                    break

    # create a new image based on the found values
    finalImage = rotatedImg[y1:y2,x1:x2]