Find most frequent row or mode of a matrix of vectors - Python / NumPy

Question

I have a numpy array of shape (?,n) that represents a vector of n-dimensional vectors.

I want to find the most frequent row.

So far it seems that the best way is to just iterate over all the entries and store a count, but it seems obscene that numpy or scipy wouldn't have something builtin to perform this task.

Answer 1

Here's an approach using NumPy views , which should be pretty efficient -

def mode_rows(a):
    a = np.ascontiguousarray(a)
    void_dt = np.dtype((np.void, a.dtype.itemsize * np.prod(a.shape[1:])))
    _,ids, count = np.unique(a.view(void_dt).ravel(), \
                                return_index=1,return_counts=1)
    largest_count_id = ids[count.argmax()]
    most_frequent_row = a[largest_count_id]
    return most_frequent_row

Sample run -

In [45]: # Let's have a random arrayb with three rows(2,4,8) and two rows(1,7)
    ...: # being duplicated. Thus, the most freequent row must be 2 here.
    ...: a = np.random.randint(0,9,(9,5))
    ...: a[4] = a[8]
    ...: a[2] = a[4]
    ...: 
    ...: a[1] = a[7]
    ...: 

In [46]: a
Out[46]: 
array([[8, 8, 7, 0, 7],
       [7, 8, 2, 6, 1],
       [2, 2, 5, 7, 6],
       [6, 5, 8, 8, 5],
       [2, 2, 5, 7, 6],
       [5, 7, 3, 6, 3],
       [2, 8, 7, 2, 0],
       [7, 8, 2, 6, 1],
       [2, 2, 5, 7, 6]])

In [47]: mode_rows(a)
Out[47]: array([2, 2, 5, 7, 6])

Answer 2

The numpy_indexed package (dsiclaimer: I am its author) has functionality that does exactly this, that works on any number of dimensions:

import numpy_indexed as npi
row = npi.mode(arr)

Under the hood, it is like Divakar's solution in terms of algorithmics and complexity, with a few more bells and whistles; see the 'weights' and 'return_indices' kwargs.

Answer 3

If you're able to use Pandas, here's one approach, which draws heavily from this answer :

import numpy as np
import pandas as pd

# generate sample data
ncol = 5
nrow = 20000
matrix = np.random.randint(0,ncol,ncol*nrow).reshape(nrow,ncol)
df = pd.DataFrame(matrix)

df.head()
   0  1  2  3  4
0  3  0  4  4  4
1  4  0  0  2  0
2  3  3  2  0  0
3  0  3  4  3  3
4  1  1  3  3  3

# count duplicated rows
(df.groupby(df.columns.tolist())
   .size()
   .sort_values(ascending=False))

Output:

0  1  2  3  4
4  2  2  1  1    17
2  2  4  2  3    16
3  2  1  2  2    15
   1  2  4  3    15
                 ..
4  1  3  0  1     1
1  2  3  0  4     1

The most frequent row is the top row of this output. The frequency count is the rightmost column.