[英]How can I count holes on Lego bricks using opencv python?
I'm working on my python project where I need to count how many holes are in each assembly of Lego brickets.我正在处理我的 python 项目,我需要计算每个乐高积木组件中有多少个孔。 Information about which assembly I need to count I will take from input .json file which looks like this:
我将从如下所示的输入 .json 文件中获取有关我需要计算哪个程序集的信息:
"img_001": [
{
"red": "0",
"blue": "2",
"white": "1",
"grey": "1",
"yellow": "1"
},
{
"red": "0",
"blue": "1",
"white": "0",
"grey": "1",
"yellow": "0"
So I need to recognize which assembly I have to count by colours.所以我需要识别我必须按颜色计算哪个组件。 Then I have to and number of holes in particular assembly of brickets.
然后我必须和特别组装砖块的孔数。
This is example of image that I work with:这是我使用的图像示例:
I've started with changing my image to hsv colour space and with using trackbar I found a mask for each colour.我开始将图像更改为 hsv 颜色空间,并使用轨迹栏为每种颜色找到了一个蒙版。 With using
cv2.inRange
I get a mask for example for red color:使用
cv2.inRange
我得到了一个面具,例如红色: As you can see reflecting light doesn't help.
如您所见,反射光无济于事。 At this point I don't know how could I move forward.
在这一点上,我不知道我该如何前进。 I feel I should use
cv2.findContour
to get contour of each assembly.我觉得我应该使用
cv2.findContour
来获取每个组件的轮廓。 I was thinking that Histogram Equalization could be useful here.我在想直方图均衡在这里可能很有用。 To detecting circles I want to use
cv2.HoughCircles
or maybe cv2.SimpleBloopDetector
.要检测圆圈,我想使用
cv2.HoughCircles
或者cv2.SimpleBloopDetector
。 But I have no idea how could I check how many brickets I have in each area.但是我不知道如何检查每个区域有多少砖。 Output is just a number of holes in particular assembly.
输出只是特定装配中的一些孔。 Could you get me some ideas?
你能给我一些想法吗? Which OpenCv function may have apply here?
哪个 OpenCv 函数可能适用于此? How would you solve this kind of image-processing problem?
你会如何解决这种图像处理问题? Thanks for your answers.
感谢您的回答。
This is a simple but very interesting exercise of color segmentation .这是一个简单但非常有趣的颜色分割练习。 This topic has been extensively covered everywhere with several examples spread around Stackoverflow .
这个话题到处都有,有几个例子围绕Stackoverflow传播。 On many scenarios, color segmentation works best in the HSV color space.
在许多情况下,颜色分割在 HSV 颜色空间中效果最好。
On the left image below you can see the segmentation result of the yellow bricks with blue-ish holes, just to show that they were also detected by this approach.在下面的左图中,您可以看到带有蓝色孔的黄色砖块的分割结果,只是为了表明它们也被这种方法检测到。
In this answer I provide a high-level overview of the operations required to detect yellow bricks and identify the holes in them.在这个答案中,我对检测黄砖并识别其中的孔所需的操作进行了高级概述。 It does not, however, demonstrates how to count the number of holes inside a particular brick to avoid spoiling your homework.
但是,它没有演示如何计算特定砖块内的孔数以避免破坏您的作业。 That part I left out of the answer on purpose to leave some work for you to do.
我故意省略了答案的那部分,以便为您留下一些工作。
Here are the main steps of my approach :以下是我的方法的主要步骤:
Convert the preprocessed image to HSV color space to achieve a better segmentation by color.将预处理后的图像转换为 HSV 颜色空间,以实现更好的颜色分割。
As this approach focus only on the segmentation of yellow bricks, the algorithm defines low and high values of yellow (in HSV) to threshold the image using this range: any color outside the range becomes black pixels.由于此方法仅关注黄砖的分割,因此该算法定义了黄色的低值和高值(在 HSV 中)以使用此范围对图像进行阈值处理:该范围之外的任何颜色都将变为黑色像素。 An image editor can help you zoom in on the original image and inspect the exact HSV values of the pixels.
图像编辑器可以帮助您放大原始图像并检查像素的准确 HSV 值。 Here is the result of the segmentation:
下面是分割的结果:
cv2.fillPoly()
and fill it with white, you'll be able to draw the entire brick without any holes in a separate image to create a mask.cv2.fillPoly()
绘制砖块的轮廓并用白色填充它,您将能够在单独的图像中绘制没有任何孔的整块砖块来创建蒙版。 This will come in handy very soon! In summary, this code offers a list of yellow bricks and another list that contains the holes in those bricks.总之,此代码提供了一个黄色砖块列表和另一个包含这些砖块中孔的列表。 From this point on it's up to you.
从现在开始,这取决于你。 The code can be easily expanded to process bricks from other colors.
该代码可以轻松扩展以处理其他颜色的砖块。 Have fun:
玩得开心:
import cv2
import numpy as np
# convertToOpenCVHSV():
# converts from HSV range (H: 0-360, S: 0-100, V: 0-100)
# to what OpenCV expects: (H: 0-179, S: 0-255, V: 0-255)
def convertToOpenCVHSV(H, S, V):
return np.array([H // 2, S * 2.55, V * 2.55], np.uint8)
# 1. Load input image
img = cv2.imread('test_images/legos.jpg')
# 2. Preprocess: quantize the image to reduce the number of colors
div = 6
img = img // div * div + div // 2
cv2.imwrite('lego2_quantized.jpg', img)
# 3. Convert to HSV color space
hsv_img = cv2.cvtColor(img, cv2.COLOR_BGR2HSV)
# 4. Segment the image using predefined values of yellow (min and max colors)
low_yellow = convertToOpenCVHSV(40, 35, 52)
high_yellow = convertToOpenCVHSV(56, 95, 93)
yellow_seg_img = cv2.inRange(hsv_img, low_yellow, high_yellow)
#cv2.imshow('yellow_seg_img', yellow_seg_img)
cv2.imwrite('lego4_yellow_seg_img.jpg', yellow_seg_img)
# 5. Identify and count the number of yellow bricks and create a mask with just the yellow objects
bricks_list = []
min_size = 5
contours, hierarchy = cv2.findContours(yellow_seg_img, cv2.RETR_EXTERNAL, cv2.CHAIN_APPROX_NONE)
for contourIdx, cnt in enumerate(contours):
# filter out tiny segments
x, y, w, h = cv2.boundingRect(cnt)
if (w < min_size) or (h < min_size):
continue
#print('contourIdx=', contourIdx, 'w=', w, 'h=', h)
bricks_list.append(cnt)
# debug: draw green contour in the original image
#cv2.drawContours(img, cnt, -1, (0, 255, 0), 2) # green
print('Detected', len(bricks_list), 'yellow pieces.')
# Iterate the list of bricks and draw them (filled) on a new image to be used as a mask
yellow_mask_img = np.zeros((img.shape[0], img.shape[1]), np.uint8)
for cnt in bricks_list:
cv2.fillPoly(yellow_mask_img, pts=[cnt], color=(255,255,255))
cv2.imshow('yellow_mask_img', yellow_mask_img)
cv2.imwrite('lego5_yellow_mask_img.jpg', yellow_mask_img)
# debug: display only the original yellow bricks found
bricks_img = cv2.bitwise_and(img, img, mask=yellow_mask_img)
#cv2.imshow('bricks_img', bricks_img)
cv2.imwrite('lego5_bricks_img.jpg', bricks_img)
# 6. Identify holes in each Lego brick
diff_img = yellow_mask_img - yellow_seg_img
cv2.imshow('diff_img', diff_img)
cv2.imwrite('lego6_diff_img.jpg', diff_img)
# debug: create new BGR image for debugging purposes
dbg_img = cv2.cvtColor(yellow_mask_img, cv2.COLOR_GRAY2RGB)
#dbg_img = bricks_img
holes_list = []
min_area_size = 10
max_area_size = 24
contours, hierarchy = cv2.findContours(yellow_seg_img, cv2.RETR_LIST, cv2.CHAIN_APPROX_NONE)
for contourIdx, cnt in enumerate(contours):
# filter out tiny segments by area
area = cv2.contourArea(contours[contourIdx])
if (area < min_area_size) or (area > max_area_size):
#print('contourIdx=', contourIdx, 'w=', w, 'h=', h, 'area=', area, '(ignored)')
#cv2.drawContours(dbg_img, cnt, -1, (0, 0, 255), 2) # red
continue
#print('contourIdx=', contourIdx, 'w=', w, 'h=', h, 'area=', area)
holes_list.append(cnt)
# debug: draw a blue-ish contour on any BGR image to show the holes of the bricks
for cnt in holes_list:
cv2.fillPoly(dbg_img, pts=[cnt], color=(255, 128, 0))
cv2.fillPoly(img, pts=[cnt], color=(255, 128, 0))
cv2.imwrite('lego6_dbg_img.jpg', dbg_img)
cv2.imwrite('lego6_img.jpg', img)
# 7. Iterate though the list of holes and associate them with a particular brick
# TODO
cv2.imshow('img', img)
cv2.imshow('dbg_img', dbg_img)
cv2.waitKey(0)
声明:本站的技术帖子网页,遵循CC BY-SA 4.0协议,如果您需要转载,请注明本站网址或者原文地址。任何问题请咨询:yoyou2525@163.com.