快速计数列表中int的出现

Question

I have a function that generates 1024 randomish ints within some known range (0 to N). I want to count the number of occurrences of each number I see.

normally I would do something like:

a = np.zeros(N+1)
for number in get_numbers():
  a[number] += 1

The problem I have is this is somewhat slow since all the accumulation is done in in python and not in a nice numpy function. Normally I wouldn't care about speed but this is done in an inner loop and the time really adds up.

I'd rather do something like

a = np.zeros(N+1)
nums = get_numbers():
a[nums] = a[nums]+1

but if there are duplicates in nums (and there could be, though the number of repeats ought to be low) then the indices with duplicates only gets counted once. Is there a faster way to do this in numpy?

Answer 1

Use np.unique with return_counts=True

a = np.array(list('aaaabbbccd'))

u, c = np.unique(a, return_counts=True)

np.column_stack([u, c])

array([['a', '4'],
       ['b', '3'],
       ['c', '2'],
       ['d', '1']], 
      dtype='<U21')

Answer 2

You can use:

np.bincount(get_numbers(), minlength=N+1)

Example :

N = 5
numbers = np.random.randint(N, size=10)
numbers
# array([2, 0, 4, 0, 0, 4, 2, 1, 2, 0])

Results using bincount :

np.bincount(numbers, minlength=N+1)
# array([4, 1, 3, 0, 2, 0])

Results using for loop:

a = np.zeros(N+1)
for number in numbers:
    a[number] += 1

a
# array([ 4.,  1.,  3.,  0.,  2.,  0.])

Timing :

N = 20
numbers = np.random.randint(N, size=1000)

def for_loop():
    a = np.zeros(N+1)
    for number in numbers:
        a[number] += 1
    return a

def np_unique():
    a = np.zeros(N+1)
    u, c = np.unique(numbers, return_counts=True)
    a[u] = c
    return a

%timeit np.bincount(numbers, minlength=N+1)
# The slowest run took 6.46 times longer than the fastest. This could mean that an intermediate result is being cached.
# 100000 loops, best of 3: 2.59 µs per loop

%timeit for_loop()
# 1000 loops, best of 3: 426 µs per loop

%timeit np_unique()
# The slowest run took 4.08 times longer than the fastest. This could mean that an intermediate result is being cached.
# 10000 loops, best of 3: 30.6 µs per loop

Checking results :

(np_unique() == np.bincount(numbers, minlength=N+1)).all()
# True

(for_loop() == np.bincount(numbers, minlength=N+1)).all()
# True

Answer 3

You can also use Counter from collections module

import numpy as np
from collections import Counter

a = np.random.randint(0, 10, 20)
c = Counter(a)
list(c.items())

Answer 4

You can use scipy's itemfreq . If you don't want to rely on scipy then define one of your own

def itemfreq(a):
    items, inv = np.unique(a, return_inverse=True)
    freq = np.bincount(inv)
    return np.array([items, freq]).T

To count number of occurrences an array you would do

In [0]: a = np.random.randint(0,100,20)
Out[0]: array([5, 8, 4, 2, 6, 8, 2, 4, 1, 3, 9, 6, 9, 2, 0, 5, 8, 8, 8, 0])
In [1]: itemfreq(a)
Out[1]: 
array([[0, 2],
   [1, 1],
   [2, 3],
   [3, 1],
   [4, 2],
   [5, 2],
   [6, 2],
   [8, 5],
   [9, 2]])

Since numpy 1.9 there is a builtin return keyword for unique function:

In [2]: np.unique(a,return_counts=True)
Out[2]: (array([0, 1, 2, 3, 4, 5, 6, 8, 9]), array([2, 1, 3, 1, 2, 2, 2, 5, 2]))

Note the numpy.unique returns sorted values.