Flipping continuous chunks of 1's up to a certain size in a binary numpy matrix

Question

I am working on an image analysis project. I have gotten my picture of interest (a NxM numpy array) to a binary format. The '1' in the matrix are regions of interest. There are regions of interest, and there is noise that cannot possibly represent features on an image. For example, in a horizontal snap of the image, isolated 1's, or groups of 2 up to, say, 5 consecutive 1's are not of interest to me. I would like to find a quick way to flip these (ie make them =0).

my MWE for flipping isolated 1's:

import numpy as np
img = np.random.choice([0,1],size=(1000,1000), p=[1./2,1./2])

#now we take the second derivative of the matrix in the horizontal axis
#since we have a binary matrix, an isolated 1, that is [...010...] is captured
#by a second derivative entry equal to -2
#because ([...010...]->dx->[...1,-1,...]->dx->[...-2...]

ddx_img = np.diff(np.diff(img,1),1)
to_flip = np.where(ddx_img==-2) #returns a tuple of [x,y] matrix entries

# the second derivative eats up an index position on horizontally, so I need to add
# +1 to the horizontal axis of the tuple

temp_copy = to_flip[1].copy() #cannot modify tuple directly, for some reason its read only
temp_copy+=1
to_flip = (to_flip[0],temp_copy)

#now we can flip the entries by adding +1 to the entries to flip and taking mod 2
img[to_flip]=mod(img[to_flip]+1,2)

This takes around 9ms on my machine. I could do with routines of up to 1 second.

I would welcome any criticism on the code ( I am not a good python programmer ), and any ideas on how to efficiently extend this procedure to eliminate isolated islands of consecutive 1s up to islands of generic size S.

Thanks in advance

edit: I realize the mod is unnecessary. At the time I did this I wanted also to flip too small islands of 0's. One could replace the =mod.... by =0

Answer 1

Question-specific case

After the edits, it seems you could use some slicing and thus avoid making intermediate copies for some performance improvement. Here's two-lines of codes to achieve the desired output -

# Calculate second derivative
ddx_img = np.diff(np.diff(img,1),1)

# Get sliced version of img excluding the first and last columns 
# and use mask with ddx elements as "-2" to zeros
img[:,1:-1][ddx_img==-2] = 0

Runtime tests and verify results -

In [42]: A = np.random.choice([0,1],size=(1000,1000), p=[1./2,1./2])

In [43]: def slicing_based(A):
    ...:    img = A.copy()
    ...:    ddx_img = np.diff(np.diff(img,1),1)
    ...:    img[:,1:-1][ddx_img==-2] = 0
    ...:    return img
    ...: 
    ...: 
    ...: def original_approach(A):
    ...: 
    ...:    img = A.copy()
    ...: 
    ...:    ddx_img = np.diff(np.diff(img,1),1)
    ...:    to_flip = np.where(ddx_img==-2)
    ...: 
    ...:    temp_copy = to_flip[1].copy()
    ...:    temp_copy+=1
    ...:    to_flip = (to_flip[0],temp_copy)
    ...: 
    ...:    img[to_flip] = 0
    ...: 
    ...:    return img
    ...: 

In [44]: %timeit slicing_based(A)
100 loops, best of 3: 15.3 ms per loop

In [45]: %timeit original_approach(A)
10 loops, best of 3: 20.1 ms per loop

In [46]: np.allclose(slicing_based(A),original_approach(A))
Out[46]: True

---

Generic case

To make the solution generic, one can use some signal processing, specifically 2D convolution as shown here -

# Define kernel
K1 = np.array([[0,1,1,0]]) # Edit this for different island lengths
K2 = 1-K1

# Generate masks of same shape as img amd based on TRUE and inverted versions of 
# kernels being convolved and those convolved sums being compared against the 
# kernel sums indicating those spefic positions have fulfiled both the ONES 
# and ZEROS criteria
mask1 = convolve2d(img, K1, boundary='fill',fillvalue=0, mode='same')==K1.sum()
mask2 = convolve2d(img==0, K2, boundary='fill',fillvalue=0, mode='same')==K2.sum()

# Use a combined mask to create that expanses through the kernel length 
# and use it to set those in img to zeros
K3 = np.ones((1,K1.size))
mask3 = convolve2d(mask1 & mask2, K3, boundary='fill',fillvalue=0, mode='same')>0
img_out = img*(~mask3)

Sample input, output -

In [250]: img
Out[250]: 
array([[0, 1, 1, 1, 0, 1, 1, 1],
       [1, 1, 1, 1, 1, 1, 0, 1],
       [1, 0, 1, 1, 1, 1, 0, 0],
       [1, 1, 1, 1, 0, 1, 0, 1],
       [1, 1, 0, 1, 1, 0, 1, 1],
       [1, 0, 1, 1, 1, 1, 1, 1],
       [1, 1, 0, 1, 1, 0, 1, 0],
       [1, 1, 1, 0, 1, 1, 1, 1]])

In [251]: img_out
Out[251]: 
array([[0, 1, 1, 1, 0, 1, 1, 1],
       [1, 1, 1, 1, 1, 1, 0, 1],
       [1, 0, 1, 1, 1, 1, 0, 0],
       [1, 1, 1, 1, 0, 1, 0, 1],
       [1, 1, 0, 0, 0, 0, 0, 1],
       [1, 0, 1, 1, 1, 1, 1, 1],
       [1, 1, 0, 0, 0, 0, 0, 0],
       [1, 1, 1, 0, 1, 1, 1, 1]])