Sorting nonzero elements of a numpy array and getting their indices

Question

I have a numpy array:

x =   numpy.array([0.1, 0, 2, 3, 0, -0.5])

I want to get an array y which contains the nonzero elements of x sorted and idx which is the corresponding indices for x.

For example for the above example y would be [3, 2, 0.1, -0.5] and idx would be [3, 2, 0, 5]. I prefer a method that can be extended to the 2d array without looping over rows of x.

For a 2d example if i have

x = [[0.1, 0, 2, 3, 0, -0.5],
     [1, 0, 0, 0, 0, 2 ]] 

i want to get a

y =[[3, 2, 0.1, -0.5],[2,1]] and 
idx = [[3, 2, 0, 5], [5, 0]].

Answer 1

Here are two vectorized approaches to solve for 1D and 2D cases separately -

def sort_nonzeros1D(x):
    sidx = np.argsort(x)
    out_idx = sidx[np.in1d(sidx, np.flatnonzero(x!=0))][::-1]
    out_x = x[out_idx]
    return out_x, out_idx

def sort_nonzeros2D(x):
    x1 = np.where(x==0, np.nan, x)
    sidx = np.argsort(x1,1)[:,::-1]

    n = x.shape[1]
    extent_idx = (x==0).sum(1)
    valid_mask = extent_idx[:,None] <= np.arange(n)
    split_idx = (n-extent_idx[:-1]).cumsum()

    out_idx = np.split(sidx[valid_mask], split_idx)
    y = x[np.arange(x.shape[0])[:,None], sidx]
    out_x = np.split(y[valid_mask], split_idx)
    return out_x, out_idx

Sample runs

1D Case :

In [461]: x
Out[461]: array([ 0.1,  0. ,  2. ,  3. ,  0. , -0.5])

In [462]: sort_nonzeros1D(x)
Out[462]: (array([ 3. ,  2. ,  0.1, -0.5]), array([3, 2, 0, 5]))

2D Case :

In [470]: x
Out[470]: 
array([[ 0.1,  0. ,  2. ,  3. ,  0. , -0.5],
       [ 1. ,  0. ,  0. ,  0. ,  0. ,  2. ],
       [ 7. ,  0. ,  2. ,  5. ,  1. ,  0. ]])

In [471]: sort_nonzeros2D(x)
Out[471]: 
([array([ 3. ,  2. ,  0.1, -0.5]),
  array([ 2.,  1.]),
  array([ 7.,  5.,  2.,  1.])],
 [array([3, 2, 0, 5]), array([5, 0]), array([0, 3, 2, 4])])

Answer 2

Here's another solution

nzidx = np.where(x)
ranking = np.argsort(x[nzidx]) # append [::-1] for descending order
result = tuple(np.array(nzidx)[:, ranking])

The elements in order can be retrieved by x[result] regardless of dimensionality.

Demo:

>> 
>>> x
array([[ 0.        , -1.36688591,  0.12606516, -1.8546047 ,  0.        ,  0.39758545],
       [ 0.65160821, -1.80074214,  0.        ,  0.        ,  1.20758375,  0.33281977]])
>>> nzidx = np.where(x)
>>> ranking = np.argsort(x[nzidx])
>>> result = tuple(np.array(nzidx)[:, ranking])
>>> 
>>> result
(array([0, 1, 0, 0, 1, 0, 1, 1]), array([3, 1, 1, 2, 5, 5, 0, 4]))
>>> x[result]
array([-1.8546047 , -1.80074214, -1.36688591,  0.12606516,  0.33281977,
        0.39758545,  0.65160821,  1.20758375])

Update:

If the sorting should be row by row we can use list comprehension

nzidx = [np.where(r)[0] for r in x]
ranking = [np.argsort(r[idx]) for r, idx in zip(x, nzidx)]
result = [idx[rk] for idx, rk in zip(nzidx, ranking)]

or

nzidx = np.where(x)
blocks = np.searchsorted(nzidx[0], np.arange(1, x.shape[0]))
ranking = [np.argsort(r) for r in np.split(x[nzidx], blocks)]
result = [idx[rk] for idx, rk in zip(np.split(nzidx[1], blocks), ranking)]

Demo:

>>> x
array([[ 0.        ,  0.        ,  0.        ,  0.        ,  0.1218789 ,
         0.        ,  0.        ,  0.        ],
       [ 0.        , -0.6445128 , -0.00603869,  1.47947823, -1.4370367 ,
         0.        ,  1.11606385, -1.22169137],
       [ 0.        ,  0.        ,  0.        ,  1.54048119, -0.85764299,
         0.        ,  0.        ,  0.32325807]])
>>> nzidx = np.where(x)
>>> blocks = np.searchsorted(nzidx[0], np.arange(1, x.shape[0]))
>>> ranking = [np.argsort(r) for r in np.split(x[nzidx], blocks)]
>>> result = [idx[rk] for idx, rk in zip(np.split(nzidx[1], blocks), ranking)]
>>> # package them
... [(r[idx], idx) for r, idx in zip(x, result)]
[(array([ 0.1218789]), array([4])), (array([-1.4370367 , -1.22169137, -0.6445128 , -0.00603869,  1.11606385,
        1.47947823]), array([4, 7, 1, 2, 6, 3])), (array([-0.85764299,  0.32325807,  1.54048119]), array([4, 7, 3]))]

Answer 3

Here's a non-numpy approach:

# create (index, value) tuple pairs for each value in `x` if value isn't 0
idxs_vals = [(idx, val) for idx, val in enumerate(x) if val != 0]

# sort the tuples from above according to the value
s_idxs_vals = sorted(idxs_vals, key = lambda x: -x[1])           

# grab the value from each tuple
y = [j for i, j in s_idxs_vals]

# grab the index from each tuple
idxs = [i for i, j in s_idxs_vals]