如何使用OpenCV检测图像帧中的对象？

Question

I am developing a rover using Raspberry Pi , that will sweep a room and pick up objects fallen on the ground . To detect the object , I am using a reference Image , that is taken right at the start of the rover's operation , and an Image (new Image) that is clicked every 10 seconds . To determine if there is a change in the image frame , I do a image subtraction between the reference image and the new image . If any difference is found , it'll draw a contour around it , and if the contour area is greater than a certain threshold (cautionary step) , it concludes that there exists an object .

I am using the following code -

import numpy as np
import cv2,time

img=cv2.imread("object1.jpg")
img1=cv2.imread("object2.jpg")
sub=cv2.subtract(img,img1)

gray=cv2.cvtColor(sub,cv2.COLOR_BGR2GRAY)
blur = cv2.GaussianBlur(gray,(3,3),0)
_, contours, _= cv2.findContours(blur,cv2.RETR_TREE,cv2.CHAIN_APPROX_SIMPLE)
c=max(contours,key=cv2.contourArea)
print(cv2.contourArea(c))

if cv2.contourArea>20000:
   print("Object detected !")

The above code just uses 2 images to calculate their difference and determine if there is an object present .Please note that I have not posted the original code here that I will be using in my project .

Now , the above code works fine for very controlled situations, say ,when the image background is very constant or there is no presence of shadow in it . But considering the fact that the rover will be moving around in the room , and there are chances that the lighting variations might trigger a false object detection , even if there is no real object in the frame . The difference might be triggered due to false contouring from shadow effects .

I want to know , if there is any other way of achieving this object detection without doing foreground/background image subtraction . I have also considered using a ultrasonic sensor to detect the object's presence , however that is not a very reliable option . I would prefer a Image processing based solution .

Thank you .

==========================================================================

EDIT 1 -

So , I decided to change the algorithm a bit . I have done thresholding on both the foreground and background Images and then performed absdiff between the binary images , to obtain any frame change(object) . The code is as follows -

import numpy as np
import cv2,time

img1=cv2.imread("back.jpeg")
blur1 = cv2.GaussianBlur(img1,(5,5),0)
gray1=cv2.cvtColor(blur1,cv2.COLOR_BGR2GRAY)
ret,thresh1 = cv2.threshold(gray1,65,255,cv2.THRESH_BINARY_INV)

img2=cv2.imread("front.jpeg")
blur2 = cv2.GaussianBlur(img2,(5,5),0)
gray2=cv2.cvtColor(blur2,cv2.COLOR_BGR2GRAY)
ret,thresh2 = cv2.threshold(gray2,65,255,cv2.THRESH_BINARY_INV)

diff=cv2.absdiff(thresh2,thresh1)
diff=cv2.bitwise_xor(diff,thresh1)

kernel = np.ones((2,2),np.uint8)
diff=cv2.erode(diff,kernel,iterations = 1)
diff=cv2.dilate(diff,kernel,iterations = 8)

_, contours, _= cv2.findContours(diff,cv2.RETR_TREE,cv2.CHAIN_APPROX_SIMPLE)
c=max(contours,key=cv2.contourArea)
x,y,w,h = cv2.boundingRect(c)
cv2.rectangle(diff,(x,y),(x+w,y+h),(125,125,125),2)


cv2.imshow("thresh",diff)
cv2.waitKey(0)

"absdiff" is followed by Erosion and Dilation . After that , I find the largest contour and determine if there is an object . The images used in the algorithm are as follows -

1. Background Image - Background Image

2. Foreground Image - Foreground image

3. Foreground Threshold - Foreground threshold Image

4. Background Threshold - Background threshold Image

5. Difference Image - Final Image with contour and its boundary .

As you can see , the detection works fine . I have few other foreground images which I have used to test the algorithm . They have given satisfactory results .I wanted to know , if there is any other way of achieving the same result with better efficiency .

PS- All of the foreground Images have been taken with Flash ON . I have tried with Flash OFF but there seems to exist a lot of noise in the image .

=============================================================

EDIT 2-

Performance of the algo using other pictures -

Note :- The Background Image has remained same .

1. Object 1 - Foreground Image 1
2. Object 1 Detection - Foreground Image 1 Result

Answer 1

I doubt if this problem is that much simple as you have described in the question, it will be get very complex when we move in real world scenario.

But anyway assuming you have small objects only present in the room then you identify them by identifying connected components in the binary image captured and choose them based on their relative pixels sizes.

Here is Python implementation for the same:

img = cv2.imread('D:/Image/objects.jpg')
gray = cv2.cvtColor(img, cv2.COLOR_BGR2GRAY)
# binarize the image
ret, bw = cv2.threshold(gray, 128, 255, 
cv2.THRESH_BINARY_INV+cv2.THRESH_OTSU)

# find connected components
connectivity = 4
nb_components, output, stats, centroids = 
cv2.connectedComponentsWithStats(bw, connectivity, cv2.CV_32S)
sizes = stats[1:, -1]; nb_components = nb_components - 1
min_size = 250 #threshhold value for objects in scene
img2 = np.zeros((img.shape), np.uint8)
for i in range(0, nb_components+1):
    # use if sizes[i] >= min_size: to identify your objects
    color = np.random.randint(255,size=3)
    # draw the bounding rectangele around each object
    cv2.rectangle(img2, (stats[i][0],stats[i][1]),(stats[i][0]+stats[i][2],stats[i][1]+stats[i][3]), (0,255,0), 2)
    img2[output == i + 1] = color

Image containing objects:

Detected objects using connected components labels:

Answer 2

Another approach you can consider is to use structure from motion, reconstruct the environment / point cloud, and the point cluster above the floor surface belongs to your object. Also could combine this with your background subtraction method to remove false detection like shadows that are causing issue for that method.