How to find the rectangles in an image

Question

This is the image generated by this code in this answer (except for the green circles, which I drew on afterwards):

The shape is (707, 1028, 3), so each pixel has three channels (RGB), but is only filled with white and black. It would be better if it were converted to an 8-bit image.

I need to get the position and the size of each rectangle in the image. I have some code using PIL and .load() to access each pixel, but is too slow. In the PIL version I look for start corner and end corner. The code is like pixels[x, y] == 255 and pixels[x-1, y] == 0 and pixels[x, y-1] == 0

Answer 1

1. Making an image with a single channel

If you need an image to have a single channel, then generate it with one channel instead of three. So instead of:

output = numpy.zeros(img.shape) # (height, width, 3)
output[~mask] = (255, 255, 255)

write:

output = numpy.zeros(img.shape[:2]) # just (height, width)
output[~mask] = 255

Or, if you have loaded a multi-channel image and want to pick just one channel for processing, slice it:

img = img[...,0] # red channel

But if you are doing further processing like feature detection, then you don't need to save an output image here or reload it. You can just carry on working with mask .

2. Finding contiguous regions

You can find contiguous regions of an image using scipy.ndimage.measurements.label . By default, this finds orthogonally connected regions only; if you want diagonally connected regions too, then pass the appropriate structure argument:

labels, n = scipy.ndimage.measurements.label(mask, numpy.ones((3, 3)))

The results are labels (an array of the same shape as mask containing different integers labelling the contiguous regions of mask ), and n (the number of regions found). You then call scipy.ndimage.measurements.find_objects to get the bounding boxes:

>>> bboxes = scipy.ndimage.measurements.find_objects(labels)
>>> bboxes[0]
(slice(0, 2, None), slice(19, 23, None))

So this object is found at x = 19–23 and y = 0–2 (it's the little sliver of black along the top edge of the image). You can get the sub-image containing the object by using this pair of slices to index the original image. Here's the uppermost of your rectangles, in object #3:

>>> bboxes[3]
(slice(33, 60, None), slice(10, 86, None))
>>> img[bboxes[3]]
array([[255, 255,   0, ...,   0, 255, 255],
       [255,   0,   0, ...,   0,   0, 255],
       [  0,   0, 255, ...,   0,   0, 255],
       ..., 
       [  0,   0,   0, ...,   0,   0, 255],
       [255,   0,   0, ...,   0, 255, 255],
       [255, 255, 255, ..., 255, 255, 255]], dtype=uint8)

(The other rectangles are objects #4, #5 and #8.) Here's one way to visualize them:

boxed = numpy.dstack((img,) * 3)
for y, x in bboxes:
    if y.stop - y.start == 27: # is this the right criterion?
        boxed[(y.start, y.stop-1), x] = (0, 255, 0)
        boxed[y, (x.start, x.stop-1)] = (0, 255, 0)
imsave('boxed.png', boxed)