我如何從他在 opencv 的背景中提取這個 object？

Question

在我的實習中，我試圖從所獲得的視覺相機鏡頭中提取這種類型的鋁線。 目的是提取這些連接並使用機器學習對其進行分類。 我的想法是提取這些連接以消除所有噪聲（背景）並分析鍵合的灰度值密度，灰度值下降 plot 將表示斷線。

我對視覺非常陌生，我試圖真正深入研究邊緣檢測和分割。 我的問題是我無法完全消除噪聲，這會導致使用 Canny 運算符進行以下邊緣檢測。 Sobel 算子會產生過多的噪聲。

這是我最近幾天取得的最好成績，希望你能幫助我在 Canny Operator 之前預處理這張圖像，但捕獲 object 的技巧也能有所幫助。 由於拍攝此視覺的空間和過程的限制，視覺相機的物理添加很困難但值得贊賞，所以仍然評論我可以做些什么來改進采集。

我的代碼：

import numpy as np
from PIL import Image
import cv2

blur = 3
canny_low = 15
canny_high = 230
min_area = 0.005
max_area = 0.025
dilate_iter = 10
erode_iter = 10
mask_color = (0.0,0.0,0.0)

image1 = cv2.imread(r"C:/Users/User/Pictures/vlcsnap-2022-11-21-15h59m05s146.png")
image2 = cv2.imread(r"C:/Users/User/Pictures/template2.png")

# function for object extraction from  background
def bgRemoval_seg(source, template):
    global blur, canny_low, canny_high, min_area, max_area, dilate_iter, erode_iter, mask_color
    # change source image and template in gray c
    source = cv2.cvtColor(source, cv2.COLOR_BGR2GRAY)
    template = cv2.cvtColor(template, cv2.COLOR_BGR2GRAY)
    # add gaussian filter for smooth blur
    source = cv2.GaussianBlur(source, (blur,blur), 0)
    template = cv2.GaussianBlur(template, (blur,blur), 0)
    # add bilateral Filter ro remove A LOT of noise ( I tried various values with this filter)
    source = cv2.bilateralFilter(source,7,100,100)
    template = cv2.bilateralFilter(template,7,100,100)
    

   # add adaptive contrast that increases the amount of contours around the object (also increases noise)
    clahe = cv2.createCLAHE(clipLimit=3.7, tileGridSize=(4,4))
    source = clahe.apply(source)
    template = clahe.apply(template)
    
    cv2.imshow("contrast", source)
    cv2.imshow("contrast2", template)
    
    # apply Canny Operator for edge detection
    edges1 = cv2.Canny(source, canny_low, canny_high)
    edges2 = cv2.Canny(template, canny_low, canny_high)
    
    #dilate and erode the image to remove more noise

    edges1 = cv2.dilate(edges1, None)
    edges2 = cv2.dilate(edges2, None)
    
    edges1 = cv2.erode(edges1, None)
    edges2 = cv2.erode(edges2, None)

    
    edges1  = np.array(edges1)
    edges2  = np.array(edges2)
    
    cv2.imshow("edges1", edges1)
    cv2.imshow("edges2", edges2)

    # get the contours and their areas
    contour_info_1 = [(c, cv2.contourArea(c),) for c in cv2.findContours(edges1, cv2.RETR_EXTERNAL, cv2.CHAIN_APPROX_NONE)[0]]
    contour_info_2 = [(c, cv2.contourArea(c),) for c in cv2.findContours(edges2, cv2.RETR_EXTERNAL, cv2.CHAIN_APPROX_NONE)[0]]
   

    # Get the area of the image as a comparison
    image_area = source.shape[0] * source.shape[1]
        
    # calculate max and min areas in terms of pixels
    max_area = max_area * image_area
    min_area = min_area * image_area

    
    # Set up mask with a matrix of 0's
    mask1 = np.zeros(edges1.shape, dtype = np.uint8)
    # Go through and find relevant contours and apply to mask
    for i in range(0,len(contour_info_1)):   
        # Instead of worrying about all the smaller contours, if the area is smaller than the min, the loop will break
        contour1 = contour_info_1[i]
        if contour1[1] > min_area and contour1[1] < max_area:
            # Add contour to mask
            mask1 = cv2.fillConvexPoly(mask1, contour1[0], (255))
        
    
    # use dilate, erode, and blur to smooth out the mask
    mask = mask1
    mask = cv2.dilate(mask, None, iterations=dilate_iter)
    mask = cv2.erode(mask, None, iterations=erode_iter)
    mask = cv2.GaussianBlur(mask, (blur,blur), 0)
    mask = np.array(mask)
    # Ensures data types match up
    mask_color = np.array(mask_color)
    mask_color = np.reshape(mask_color,[1,3])
    mask = mask.astype('float32') / 255.0           
    source= source.astype('float32') / 255.0
    # Blend the image and the mask
    masked = (mask * source)
    masked = (masked * 255).astype('uint8')

    return masked




while(True):
    
    # Get Region of interest
    x,y,w,h = cv2.selectROI(image1)
    # Recommended values for the crop
    # X: 145 , Y: 292 , W: 1035 , H: 445 
   
    # Crop image and use same crop for template
    imageCrop1 = image1[int(y):int(y+h), int(x):int(x+w)]
    
    print(x,y,w,h)
    
    imageCrop2 = image2[int(y):int(y+h), int(x):int(x+w)]
    
    # Display the resulting frame
    
    cv2.imshow("Foreground Canny ",bgRemoval_seg(imageCrop1, imageCrop2))

    if cv2.waitKey(1) & 0xFF == ord('q'):
        break

# When everything done, release the capture 

cv2.destroyAllWindows()

原圖：原圖

過濾后（First is original, second is template, how it needs to look）： 原始： bilateral filters and adaptive contrast filter

模板：模板雙邊濾波器和自適應對比濾波器

Canny Operation: 原創：很多由陰影引起的噪點也有背景噪點Template（終極目標）： template canny operation

Object 從后台提取：最終結果

我希望你可以幫助我！

我已經嘗試過不同的過濾器。 此外，我嘗試了 Grabcut 算法。 我也做了一些閾值處理，但很早就停止了，沒有深入研究。 我也嘗試用高斯濾波器除法，但結果保持不變。

Answer 1

如評論中所述，這是一個棘手的問題，因為圖像有很多您不關心的邊緣，而且也很難按顏色過濾。 但是，我認為有一個功能可能會有所幫助，那就是模糊。 具體來說，電線是焦點，而鏡頭的rest不是。

您可以使用拉普拉斯濾波器利用這一事實。 拉普拉斯濾波器通常用於通過查看濾波器過零的位置來檢測邊緣。 但是，它也可用於通過查找過濾器值在大范圍內較小的區域來檢測模糊。 為了獲得整條線，我在拉普拉斯濾波器之后使用了高斯平滑濾波器，它會在整個線的寬度上塗抹高值。 然后，該值被閾值化。

import cv2
import matplotlib.pyplot as plt
import numpy as np
import scipy.ndimage

image = cv2.imread('test192_img.png')

laplacian_spread_distance = 15  # distance to spread laplacian in pixels
wire_threshold = 110  # Out of 255. Higher values mean less of the image is kept.

gray = cv2.cvtColor(image, cv2.COLOR_BGR2GRAY)
fm = np.abs(cv2.Laplacian(gray, cv2.CV_64F))
fm = scipy.ndimage.gaussian_filter(fm, sigma=lp_spread_distance)
fm /= fm.max() / 255
fm = fm.astype('uint8')
ret2, thresholded = cv2.threshold(fm, wire_threshold, 1, cv2.THRESH_BINARY)
extracted = thresholded.reshape(image.shape[:2] + (1,)) * image

Output 來自這個過濾器：

此方法假定電線處於焦點位置。 如果你有一個自動對焦相機，或者如果相機和電線之間的距離在變化，那么這個假設可能是不合理的。