使用Sci-Kit对调整大小的图像进行聚类

Question

I have several product images of different sizes and I am running a clustering algorithm in sci-kit learn to group similar images together. The images vary in size, but are generally around 500x500, I am shrinking them down to 250x250

def read_img(path, mode='L', size_one=(250, 250)):
    return misc.imresize(misc.imread(path, mode=mode), size_one)

Once I get the vector, I remove white borders from it and flatten the vector.

The issue is that some images that are visually very similar have different quality (due to size before resizing). They are not getting picked up as separate clusters.

For example these two images, although very similar, have slightly different quality and don't get clustered together.

• I've tried to play around with parameters for AffinityPropagation and Kmeans cluster in sci-kit learn, but it still doesn't help.
• There is a large difference between these two vectors as well

What can I do from a pre-clustering standpoint to improve this? I am just getting started with this and any feedback would be very helpful.

Thanks in advance

EDIT: Here is how I am trimming the borders, a better approach would be very welcomed as well.

def trim_img_border(img):
    shape = img.shape
    temp_rows = []
    for row in img:
        if check_row(row):
            temp_rows.append(row)
    temp_rows_T = np.transpose(np.array(temp_rows))
    out = []
    for row in temp_rows_T:
        if check_row(row):
            out.append(row)
    return round_img(misc.imresize(np.transpose(np.array(out)), shape))


def check_row(row):
    srow = sorted(list(set(row)))
    if srow == [255] or srow == [254, 255] or srow == [253, 254, 255]:
        return False
    return True

Answer 1

The primary reason your clustering is finding different clusters is that you are doing clustering on raw pixels, rather than on features from the pixels. Pixels have a lot of variability even when visually, things look similar. So I think you have two approaches:

1. Make your pixels less variable. A few ways to do this include: slightly blur every image (with a relatively small kernel, maybe 3x3 or 5x5) before passing it to the clustering algorithm. Alternatively, or additionally, you can convert your images into HSV color space and use only the H channel to cluster.
2. Cluster based on image features rather than pixels. You can try the BRIEF or ORB features from scikit-image: http://scikit-image.org/docs/dev/api/skimage.feature.html

Hope something like this helps - would love to hear how it goes.