选择包含所有给定值的numpy数组行

Question

I have a np.array:

matrix = np.array([['A', 'B', 'C'], ['A', 'B', np.nan], ['C', np.nan, np.nan] ])

and I want to efficiently select all the rows that contains the given values

samples = ['C', 'A']

but when I make:

mask = np.isin(matrix, samples)

I get

array([[ True, False,  True],
       [ True, False, False],
       [ True, False, False]])

How can I efficiently get a mask when is True only in the rows that contains both values?

I focus on efficiently because its a sparse and big matrix.

thank you in advance estimates.

Answer 1

My first approach would be

[np.isin(samples, row).all() for row in matrix]
# [True, False, False]

(But to be honest not able to tell anything about efficiency or performance...)

Answer 2

If you want something vectorized, I would suggest doing this comparison by transforming this into 3D and broadcasting over the third dimension. Then for each slice, check each row to see if there is anything that is True . Finally, if we see that for each row, every element is True , then this is the result we should return.

In [40]: matrix = np.array([['A', 'B', 'C'], ['A', 'B', np.nan], ['C', np.nan, np.nan] ])

In [41]: samples = ['C', 'A']

In [42]: samples = np.array(samples)

In [43]: mask = matrix[...,None] == samples[None,None]

In [44]: mask
Out[44]:
array([[[False,  True],
        [False, False],
        [ True, False]],

       [[False,  True],
        [False, False],
        [False, False]],

       [[ True, False],
        [False, False],
        [False, False]]])

In [45]: mask = np.any(mask, axis=1)

In [46]: mask
Out[46]:
array([[ True,  True],
       [False,  True],
       [ True, False]])

In [47]: mask = np.all(mask, axis=1)

In [48]: mask
Out[48]: array([ True, False, False])

To do this in more shortly:

# Define data
matrix = np.array([['A', 'B', 'C'], ['A', 'B', np.nan], ['C', np.nan, np.nan] ])
samples = ['C', 'A']

# Solution
mask = np.all(np.any(matrix[...,None] == np.array(samples)[None,None], axis=1), axis=1)

Take note that this will probably not do well with large sparse matrices....

Answer 3

Here is pseudocode that might help you:

idxRows = []
for idx, i in enumerate(mask):
    if True in i:
        idxRows.append(idx)

This will give you the indices of all the rows that contain said samples.

Answer 4

I finally use:

#Filter
test_elements = ['A', 'B']
mask = np.isin(matrix, test_elements)
vec_mask = np.isin(mask.sum(axis=1), [len(test_elements)])
ids = np.where(vec_mask)
existence = matrix[ids]

Thank you for you help guys.