在python opencv中查找骨架图像的交集

Question

I have a skeletonised image (shown below).

I would like to get the intersections of the lines. I have tried the following method below, skeleton is a openCV image and the algorithm returns a list of coordinates:

def getSkeletonIntersection(skeleton):
    image = skeleton.copy();
    image = image/255;
    intersections = list();
    for y in range(1,len(image)-1):
        for x in range(1,len(image[y])-1):
            if image[y][x] == 1:
                neighbourCount = 0;
                neighbours = neighbourCoords(x,y);
                for n in neighbours:
                    if (image[n[1]][n[0]] == 1):
                        neighbourCount += 1;
                if(neighbourCount > 2):
                    print(neighbourCount,x,y);
                    intersections.append((x,y));
    return intersections;

It finds the coordinates of white pixels where there are more than two adjacent pixels. I thought that this would only return corners but it does not - it returns many more points.

This is the output with the points it detects marked on the image. This is because it detects some of the examples shown below that are not intersections.

0 0 0    1 1 0    0 1 1
1 1 1    0 1 0    1 1 0
0 0 1    0 0 1    0 0 0

And many more examples. Is there another method I should look at to detect intersections. All input and ideas appreciated, thanks.

Answer 1

I am not sure about OpenCV features, but you should maybe try using Hit and Miss morphology which is described here .

Read up on Line Junctions and see the 12 templates you need to test for:

Answer 2

I received an email recently asking for my eventual solution to the problem. It is posted below such that it could inform others. I make no claim that this code is particularly fast or stable - only that it's what worked for me! The function also includes filtering of duplicates and intersections detected too close together, suggesting that they are not real intersections and instead introduced noise from the skeletonisation process.

def neighbours(x,y,image):
    """Return 8-neighbours of image point P1(x,y), in a clockwise order"""
    img = image
    x_1, y_1, x1, y1 = x-1, y-1, x+1, y+1;
    return [ img[x_1][y], img[x_1][y1], img[x][y1], img[x1][y1], img[x1][y], img[x1][y_1], img[x][y_1], img[x_1][y_1] ]   


def getSkeletonIntersection(skeleton):
    """ Given a skeletonised image, it will give the coordinates of the intersections of the skeleton.

    Keyword arguments:
    skeleton -- the skeletonised image to detect the intersections of

    Returns: 
    List of 2-tuples (x,y) containing the intersection coordinates
    """
    # A biiiiiig list of valid intersections             2 3 4
    # These are in the format shown to the right         1 C 5
    #                                                    8 7 6 
    validIntersection = [[0,1,0,1,0,0,1,0],[0,0,1,0,1,0,0,1],[1,0,0,1,0,1,0,0],
                         [0,1,0,0,1,0,1,0],[0,0,1,0,0,1,0,1],[1,0,0,1,0,0,1,0],
                         [0,1,0,0,1,0,0,1],[1,0,1,0,0,1,0,0],[0,1,0,0,0,1,0,1],
                         [0,1,0,1,0,0,0,1],[0,1,0,1,0,1,0,0],[0,0,0,1,0,1,0,1],
                         [1,0,1,0,0,0,1,0],[1,0,1,0,1,0,0,0],[0,0,1,0,1,0,1,0],
                         [1,0,0,0,1,0,1,0],[1,0,0,1,1,1,0,0],[0,0,1,0,0,1,1,1],
                         [1,1,0,0,1,0,0,1],[0,1,1,1,0,0,1,0],[1,0,1,1,0,0,1,0],
                         [1,0,1,0,0,1,1,0],[1,0,1,1,0,1,1,0],[0,1,1,0,1,0,1,1],
                         [1,1,0,1,1,0,1,0],[1,1,0,0,1,0,1,0],[0,1,1,0,1,0,1,0],
                         [0,0,1,0,1,0,1,1],[1,0,0,1,1,0,1,0],[1,0,1,0,1,1,0,1],
                         [1,0,1,0,1,1,0,0],[1,0,1,0,1,0,0,1],[0,1,0,0,1,0,1,1],
                         [0,1,1,0,1,0,0,1],[1,1,0,1,0,0,1,0],[0,1,0,1,1,0,1,0],
                         [0,0,1,0,1,1,0,1],[1,0,1,0,0,1,0,1],[1,0,0,1,0,1,1,0],
                         [1,0,1,1,0,1,0,0]];
    image = skeleton.copy();
    image = image/255;
    intersections = list();
    for x in range(1,len(image)-1):
        for y in range(1,len(image[x])-1):
            # If we have a white pixel
            if image[x][y] == 1:
                neighbours = neighbours(x,y,image);
                valid = True;
                if neighbours in validIntersection:
                    intersections.append((y,x));
    # Filter intersections to make sure we don't count them twice or ones that are very close together
    for point1 in intersections:
        for point2 in intersections:
            if (((point1[0] - point2[0])**2 + (point1[1] - point2[1])**2) < 10**2) and (point1 != point2):
                intersections.remove(point2);
    # Remove duplicates
    intersections = list(set(intersections));
    return intersections;

This is also available on github here .

Answer 3

It might help if when for a given pixel, instead of counting the number of total 8-neighbors (= neighbors with a connectivity 8), you count the number of 8-neighbors which are not 4-neighbors with each other

So in your example of false positives

0 0 0    1 1 0    0 1 1
1 1 1    0 1 0    1 1 0
0 0 1    0 0 1    0 0 0

For every case, you have 3 neighbors, but each time, 2 of them are 4-connected. (pixels marked "2" in next snippet)

0 0 0    2 2 0    0 2 2
1 1 2    0 1 0    1 1 0
0 0 2    0 0 1    0 0 0

If you consider only one of these for your counts (instead of both of them in your code right now), you indeed have only 2 total newly-defined "neighbors" and the considered points are not considered intersections. Other "real intersections" would still be kept, like the following

0 1 0    0 1 0    0 1 0
1 1 1    0 1 0    1 1 0
0 0 0    1 0 1    0 0 1

which still have 3 newly-defined neighbors.

I haven't checked on your image if it works perfectly, but I had implemented something like this for this problem a while back...

Answer 4

Here is my solution:

    # Functions to generate kernels of curve intersection 

    def generate_nonadjacent_combination(input_list,take_n):
        """ 
        It generates combinations of m taken n at a time where there is no adjacent n.
        INPUT:
            input_list = (iterable) List of elements you want to extract the combination 
            take_n =     (integer) Number of elements that you are going to take at a time in
                         each combination
        OUTPUT:
            all_comb =   (np.array) with all the combinations
        """
        all_comb = []
        for comb in itertools.combinations(input_list, take_n):
            comb = np.array(comb)
            d = np.diff(comb)
            fd = np.diff(np.flip(comb))
            if len(d[d==1]) == 0 and comb[-1] - comb[0] != 7:
                all_comb.append(comb)        
                print(comb)
        return all_comb


    def populate_intersection_kernel(combinations):
        """
        Maps the numbers from 0-7 into the 8 pixels surrounding the center pixel in
        a 9 x 9 matrix clockwisely i.e. up_pixel = 0, right_pixel = 2, etc. And 
        generates a kernel that represents a line intersection, where the center 
        pixel is occupied and 3 or 4 pixels of the border are ocuppied too.
        INPUT:
            combinations = (np.array) matrix where every row is a vector of combinations
        OUTPUT:
            kernels =      (List) list of 9 x 9 kernels/masks. each element is a mask.
        """
        n = len(combinations[0])
        template = np.array((
                [-1, -1, -1],
                [-1, 1, -1],
                [-1, -1, -1]), dtype="int")
        match = [(0,1),(0,2),(1,2),(2,2),(2,1),(2,0),(1,0),(0,0)]
        kernels = []
        for n in combinations:
            tmp = np.copy(template)
            for m in n:
                tmp[match[m][0],match[m][1]] = 1
            kernels.append(tmp)
        return kernels


    def give_intersection_kernels():
        """
        Generates all the intersection kernels in a 9x9 matrix.
        INPUT:
            None
        OUTPUT:
            kernels =      (List) list of 9 x 9 kernels/masks. each element is a mask.
        """
        input_list = np.arange(8)
        taken_n = [4,3]
        kernels = []
        for taken in taken_n:
            comb = generate_nonadjacent_combination(input_list,taken)
            tmp_ker = populate_intersection_kernel(comb)
            kernels.extend(tmp_ker)
        return kernels


    # Find the curve intersections
    def find_line_intersection(input_image, show=0):
        """
        Applies morphologyEx with parameter HitsMiss to look for all the curve 
        intersection kernels generated with give_intersection_kernels() function.
        INPUT:
            input_image =  (np.array dtype=np.uint8) binarized m x n image matrix
        OUTPUT:
            output_image = (np.array dtype=np.uint8) image where the nonzero pixels 
                           are the line intersection.
        """
        kernel = np.array(give_intersection_kernels())
        output_image = np.zeros(input_image.shape)
        for i in np.arange(len(kernel)):
            out = cv2.morphologyEx(input_image, cv2.MORPH_HITMISS, kernel[i,:,:])
            output_image = output_image + out
        if show == 1:
            show_image = np.reshape(np.repeat(input_image, 3, axis=1),(input_image.shape[0],input_image.shape[1],3))*255
            show_image[:,:,1] = show_image[:,:,1] -  output_image *255
            show_image[:,:,2] = show_image[:,:,2] -  output_image *255
            plt.imshow(show_image)
        return output_image

    #  finding corners
    def find_endoflines(input_image, show=0):
        """
        """
        kernel_0 = np.array((
                [-1, -1, -1],
                [-1, 1, -1],
                [-1, 1, -1]), dtype="int")

        kernel_1 = np.array((
                [-1, -1, -1],
                [-1, 1, -1],
                [1,-1, -1]), dtype="int")

        kernel_2 = np.array((
                [-1, -1, -1],
                [1, 1, -1],
                [-1,-1, -1]), dtype="int")

        kernel_3 = np.array((
                [1, -1, -1],
                [-1, 1, -1],
                [-1,-1, -1]), dtype="int")

        kernel_4 = np.array((
                [-1, 1, -1],
                [-1, 1, -1],
                [-1,-1, -1]), dtype="int")

        kernel_5 = np.array((
                [-1, -1, 1],
                [-1, 1, -1],
                [-1,-1, -1]), dtype="int")

        kernel_6 = np.array((
                [-1, -1, -1],
                [-1, 1, 1],
                [-1,-1, -1]), dtype="int")

        kernel_7 = np.array((
                [-1, -1, -1],
                [-1, 1, -1],
                [-1,-1, 1]), dtype="int")

        kernel = np.array((kernel_0,kernel_1,kernel_2,kernel_3,kernel_4,kernel_5,kernel_6, kernel_7))
        output_image = np.zeros(input_image.shape)
        for i in np.arange(8):
            out = cv2.morphologyEx(input_image, cv2.MORPH_HITMISS, kernel[i,:,:])
            output_image = output_image + out

        if show == 1:
            show_image = np.reshape(np.repeat(input_image, 3, axis=1),(input_image.shape[0],input_image.shape[1],3))*255
            show_image[:,:,1] = show_image[:,:,1] -  output_image *255
            show_image[:,:,2] = show_image[:,:,2] -  output_image *255
            plt.imshow(show_image)    

        return output_image#, np.where(output_image == 1)

    # 0- Find end of lines
    input_image = img.astype(np.uint8) # must be blaack and white thin network image
    eol_img = find_endoflines(input_image, 0)

    # 1- Find curve Intersections
    lint_img = find_line_intersection(input_image, 0)

    # 2- Put together all the nodes
    nodes = eol_img + lint_img
    plt.imshow(nodes)