Inserting rows of zeros at specific places along the rows of a NumPy array

Question

I have a two column numpy array. I want to go through each row of the 2nd column, and take the difference between each set of 2 numbers (9.6-0, 19.13-9.6, etc). If the difference is > 15, I want to insert a row of 0s for both columns. I really only need to end up with values in the first column (I only need the second to determine where to put 0s), so if it's easier to split them up that would be fine.

This is my input array:

 [[0.00 0.00]
 [1.85 9.60]
 [2.73 19.13]
 [0.30 28.70]
 [2.64 38.25]
 [2.29 47.77]
 [2.01 57.28]
 [2.61 66.82]
 [2.20 76.33]
 [2.49 85.85]
 [2.55 104.90]
 [2.65 114.47]
 [1.79 123.98]
 [2.86 133.55]]

and it should turn into:

 [[0.00 0.00]
 [1.85 9.60]
 [2.73 19.13]
 [0.30 28.70]
 [2.64 38.25]
 [2.29 47.77]
 [2.01 57.28]
 [2.61 66.82]
 [2.20 76.33]
 [2.49 85.85]
 [0.00 0.00]
 [2.55 104.90]
 [2.65 114.47]
 [1.79 123.98]
 [2.86 133.55]]

Answer 1

You can do in a one liner using ediff1d , argmax and insert from numpy :

np.insert(arr, np.argmax(np.append(False, np.ediff1d(arr[:,1])>15)), 0, axis=0)

#array([[   0.  ,    0.  ],
#       [   1.85,    9.6 ],
#       [   2.73,   19.13],
#       [   0.3 ,   28.7 ],
#       [   2.64,   38.25],
#       [   2.29,   47.77],
#       [   2.01,   57.28],
#       [   2.61,   66.82],
#       [   2.2 ,   76.33],
#       [   2.49,   85.85],
#       [   0.  ,    0.  ],
#       [   2.55,  104.9 ],
#       [   2.65,  114.47],
#       [   1.79,  123.98],
#       [   2.86,  133.55]])

Answer 2

Assuming A as the input array, here's a vectorized approach based on initialization with zeros -

# Get indices at which such diff>15 occur 
cut_idx = np.where(np.diff(A[:,1]) > 15)[0]

# Initiaize output array
out = np.zeros((A.shape[0]+len(cut_idx),2),dtype=A.dtype)

# Get row indices in the output array at which rows from A are to be inserted.
# In other words, avoid rows to be kept as zeros. Finally, insert rows from A.
idx = ~np.in1d(np.arange(out.shape[0]),cut_idx + np.arange(1,len(cut_idx)+1))
out[idx] = A

Sample input, output -

In [50]: A  # Different from the one posted in question to show variety
Out[50]: 
array([[   0.  ,    0.  ],
       [   1.85,    0.6 ],
       [   2.73,   19.13],
       [   2.2 ,   76.33],
       [   2.49,   85.85],
       [   2.55,  104.9 ],
       [   2.65,  114.47],
       [   1.79,  163.98],
       [   2.86,  169.55]])

In [51]: out
Out[51]: 
array([[   0.  ,    0.  ],
       [   1.85,    0.6 ],
       [   0.  ,    0.  ],
       [   2.73,   19.13],
       [   0.  ,    0.  ],
       [   2.2 ,   76.33],
       [   2.49,   85.85],
       [   0.  ,    0.  ],
       [   2.55,  104.9 ],
       [   2.65,  114.47],
       [   0.  ,    0.  ],
       [   1.79,  163.98],
       [   2.86,  169.55]])

Answer 3

a=[[0.00, 0.00],
 [1.85, 9.60],
 [2.73, 19.13],
 [0.30, 28.70],
 [2.64, 38.25],
 [2.29, 47.77],
 [2.01, 57.28],
 [2.61, 66.82],
 [2.20, 76.33],
 [2.49, 85.85],
 [2.55, 104.90],
 [2.65, 114.47],
 [1.79, 123.98],
 [2.86, 133.55]]

i=0
while i <len(a)-1:
    if (a[i+1][1]-a[i][1])>15:
        a.insert(i+1,[0,0])
        i=i+1
    i=i+1    

for line in a :
    print line

output:

[0.0, 0.0]
[1.85, 9.6]
[2.73, 19.13]
[0.3, 28.7]
[2.64, 38.25]
[2.29, 47.77]
[2.01, 57.28]
[2.61, 66.82]
[2.2, 76.33]
[2.49, 85.85]
[0, 0]
[2.55, 104.9]
[2.65, 114.47]
[1.79, 123.98]
[2.86, 133.55]

Answer 4

Here's right algorithm:

arr = [ ... ]
result = []
result.append(arr[0])
for i in range(1, len(arr)):
    if arr[i][1] - arr[i-1][1] > 15:
        result.append([0.0,0.0])
    result.append(arr[i])

print(result)

Answer 5

A one liner that can handle more than one fill slot. Here I'm testing it on the OP example, with one modified value.

In [70]: np.insert(a, np.where(np.diff(a[:,1])>15)[0]+2,0, axis=0)


In [71]: Out[70]: 
array([[   0.  ,    0.  ],
       [   1.85,    9.6 ],
       [   2.73,   19.13],
       [   0.3 ,   28.7 ],
       [   2.64,   38.25],
       [   2.29,  140.  ],   # modified
       [   0.  ,    0.  ],
       [   2.01,   57.28],
       [   2.61,   66.82],
       [   2.2 ,   76.33],
       [   2.49,   85.85],
       [   2.55,  104.9 ],
       [   0.  ,    0.  ],
       [   2.65,  114.47],
       [   1.79,  123.98],
       [   2.86,  133.55]])

The use of where instead of argmax ( Colonel's answer) handles more than one slot. The +2 is required because diff is one short, and we are inserting after. ediff1d has more options for handling the end points.

np.insert has various strategies for filling. In this case it probably is doing something similar to Divakar's answer - create an out , and copy values to the correct slots.

Another answer uses np.abs() . That might be needed, but in my example that would add another 0 row, after the 140 drops back to 57 .

Answer 6

I'd be surprised if numpy didn't have some native methods for doing this sort of thing but I think this will work too:

i = 1
while i < len(lst):
    if abs(lst[i][1] - lst[i-1][1]) > 15:
        lst[i] = [0.0, 0.0]
        # uncomment to change only the second column
        # lst[i][1] = 0.0
        i += 1
    i += 1

Output:

>>> lst
array([[   0.  ,    0.  ],
       [   1.85,    9.6 ],
       [   2.73,   19.13],
       [   0.3 ,   28.7 ],
       [   2.64,   38.25],
       [   2.29,   47.77],
       [   2.01,   57.28],
       [   2.61,   66.82],
       [   2.2 ,   76.33],
       [   2.49,   85.85],
       [   2.55,  104.9 ],
       [   2.65,  114.47],
       [   1.79,  123.98],
       [   2.86,  133.55]])
>>> 
>>> i = 1
>>> while i < len(lst):
...     if abs(lst[i][1] - lst[i-1][1]) > 15:
...         lst[i] = [0.0, 0.0]
...         i += 1
...     i += 1
... 
>>> lst
array([[   0.  ,    0.  ],
       [   1.85,    9.6 ],
       [   2.73,   19.13],
       [   0.3 ,   28.7 ],
       [   2.64,   38.25],
       [   2.29,   47.77],
       [   2.01,   57.28],
       [   2.61,   66.82],
       [   2.2 ,   76.33],
       [   2.49,   85.85],
       [   0.  ,    0.  ],
       [   2.65,  114.47],
       [   1.79,  123.98],
       [   2.86,  133.55]])
>>>