OpenCV：如何使用色度键和轮廓检测来检测和识别纯色背景上的对象

Question

I have a camera (calibrated, warped with OpenCV) facing a flat surface top down. The surface will be a single solid color.

I want to be able to identify objects lying on the surface and determine what they are. The objects are metal parts varying from 15mm to 100mm made of the same alloy (so same color). The parts will generally be placed top down on the surface, but handling lying parts would be a nice to have. What I have so far:

1. Contour detection: (see attached image) is effective but very unreliable and jittery. Also cannot differentiate between similar sized parts or detect same part but rotated.
2. Template detection: might work but does not seem terribly robust.
3. Cascade detection and custom cascade classifier: seems like this would work, but it just feels like overkill as my objects will always be on a solid contrasting background and seen from more or less the same camera angle.

What my future setup should look like:

I want the algorithm to identify all 3 screws as screws in the easiest way possible, and return their outline in pixels. I also want it to be able to differentiate between completely different screws or parts of similar colour and size.

I am a noob in OpenCV, so I really would like to hear techniques with pro's and cons from people with experience. How would you approach this problem effectively?

Answer 1

Uncertain why this got downvoted, but after a week of experimenting, I found a really good way to find objects on a solid background in OpenCV:

1. Use chroma key correction to remove the background with function cv2.inRange

2. Invert colours so that found objects are white and everything else is black with image = cv2.bitwise_not(image)

3. Use contour detection with the cv2.RETR_LIST, cv2.CHAIN_APPROX_NONE parameter as the 'EXTERNAL' parameter will not work: cv2.findContours(img, cv2.RETR_LIST, cv2.CHAIN_APPROX_NONE)

4. return both the area (number of pixels) and the bounding box of all found contours. The combination of these is actually very powerful for identifying most 2D objects in my case.

5. If you have an aruco tag or other element with known real world mm sizes in your picture, it becomes easy to convert the area to mm2. Function for finding an aruco tag:

   arucoDict = cv2.aruco.Dictionary_get(cv2.aruco.DICT_4X4_50) arucoParams = cv2.aruco.DetectorParameters_create() (corners, ids, rejected) = cv2.aruco.detectMarkers(image, arucoDict, parameters=arucoParams)

import cv2
import numpy as np

def getContours(img,imgContour):

""" DRAWS AND FINDS CONTOURS, THEN RETURNS a list of lists incl x0, y0, w, h"""

contour_list = []
contours, hierarchy = cv2.findContours(img, cv2.RETR_LIST, cv2.CHAIN_APPROX_NONE)
# print('contours:', contours)
for cnt in contours:
    area = cv2.contourArea(cnt)
    areaMin = 200
    if area > areaMin and area < 5000:
        cv2.drawContours(imgContour, cnt, -1, (255, 0, 0), 7)
        peri = cv2.arcLength(cnt, True)
        approx = cv2.approxPolyDP(cnt, 0.02 * peri, True)
        # print(len(approx))
        x , y , w, h = cv2.boundingRect(approx)
        print('contour bounding:', x,y,w,h)
        center_x = int(x + w/2)
        center_y = int(y + h/2)

        cv2.circle(imgContour,(center_x, center_y), 5, (0, 0, 255), 5)
        cv2.rectangle(imgContour, (x , y ), (x + w , y + h ), (0, 255, 0), 5)

        cv2.putText(imgContour, "Points: " + str(len(approx)), (x + w + 20, y + 20), cv2.FONT_HERSHEY_COMPLEX, .7,
                    (0, 255, 0), 2)
        cv2.putText(imgContour, "Area: " + str(int(area)), (x + w + 20, y + 45), cv2.FONT_HERSHEY_COMPLEX, 0.7,
                    (0, 255, 0), 2)

        if area < 3500:
            cv2.putText(imgContour, "THIS IS A SMALL PART" , (x + w + 20, y + 70), cv2.FONT_HERSHEY_COMPLEX, 0.7,
                        (0, 255, 0), 2)

        contour_list.append([x,y,w,h])

return contour_list

while True:

img = cv2.imread('yellowcropped.jpg', 1)
image_original = img.copy()

lower_blue = np.array([160, 140, 15])  ##[R value, G value, B value]
upper_blue = np.array([255, 235, 60])


image_copy = np.copy(img)
image_copy = cv2.cvtColor(image_copy, cv2.COLOR_BGR2RGB)

mask = cv2.inRange(image_copy, lower_blue, upper_blue)
mask = cv2.bitwise_not(mask)
img = cv2.cvtColor(mask, cv2.COLOR_BGR2RGB)


# img = image
imgContour = image_original.copy()

getContours(mask,imgContour)
imgStack = stackImages(0.6,([mask],[imgContour]))

# cv2.imshow("Result", imgStack)
cv2.imshow('Img_contour', imgContour)
# cv2.setMouseCallback("Img_contour", mouse_callback)
if cv2.waitKey(1) & 0xFF == ord('q'):
    break