在大型numpy数组中查找常量子数组

Question

I have a numpy float array like

v = np.array([1.0,1.0,2.0,2.0,2.0,2.0,...])

I would need to identify all the constant segments in the array like

[{value:1.0,location:0,duration:2},..]

Efficiency is the main metric

Answer 1

Here's one approach -

def island_props(v):
    # Get one-off shifted slices and then compare element-wise, to give
    # us a mask of start and start positions for each island.
    # Also, get the corresponding indices.
    mask = np.concatenate(( [True], v[1:] != v[:-1], [True] ))
    loc0 = np.flatnonzero(mask)

    # Get the start locations
    loc = loc0[:-1]

    # The values would be input array indexe by the start locations.
    # The lengths woul be the differentiation between start and stop indices.
    return v[loc], loc, np.diff(loc0)

Sample run -

In [143]: v
Out[143]: array([ 1.,  1.,  2.,  2.,  2.,  2.,  5.,  2.])

In [144]: value, location, lengths = island_props(v)

In [145]: value
Out[145]: array([ 1.,  2.,  5.,  2.])

In [146]: location
Out[146]: array([0, 2, 6, 7])

In [147]: lengths
Out[147]: array([2, 4, 1, 1])

Runtime test

Other approaches -

import itertools
def MSeifert(a):
    return [{'value': k, 'duration': len(list(v))} for k, v in 
             itertools.groupby(a.tolist())]

def Kasramvd(a):
    return np.split(v, np.where(np.diff(v) != 0)[0] + 1)

Timings -

In [156]: v0 = np.array([1.0,1.0,2.0,2.0,2.0,2.0,5.0,2.0])

In [157]: v = np.tile(v0,10000)

In [158]: %timeit MSeifert(v)
     ...: %timeit Kasramvd(v)
     ...: %timeit island_props(v)
     ...: 
10 loops, best of 3: 44.7 ms per loop
10 loops, best of 3: 36.1 ms per loop
10000 loops, best of 3: 140 µs per loop

Answer 2

You can group the equal items as following then simply do the rest by getting the size of the array, first element and the index:

In [2]: v = np.array([1.0,1.0,2.0,2.0,2.0,2.0,3.0, 3.0, 5.0, 6.0, 6.0])

In [4]: np.split(v, np.where(np.diff(v) != 0)[0] + 1)
Out[4]: 
[array([ 1.,  1.]),
 array([ 2.,  2.,  2.,  2.]),
 array([ 3.,  3.]),
 array([ 5.]),
 array([ 6.,  6.,  6.])]

The equation np.diff(v) != 0 denotes the places of where the sequence changes (the difference is not 0) and np.where() gives you the respective indices of those places (from the boolean result). Then you can simply split the array using np.split() .

And finally you can use a list comprehension to get the desire result:

In [7]: locations = np.where(np.diff(v) != 0)[0] + 1

In [8]: result = np.split(v, locations)

In [9]: [{'value':arr[0], 'location':loc, 'duration':arr.size} for loc, arr in zip(locations, result)]
Out[9]: 
[{'duration': 2, 'value': 1.0, 'location': 2},
 {'duration': 4, 'value': 2.0, 'location': 6},
 {'duration': 2, 'value': 3.0, 'location': 8},
 {'duration': 1, 'value': 5.0, 'location': 9}]

Answer 3

You could use itertools.groupby , it could be a bit slower (haven't timed it) but probably a lot easier to understand:

>>> import numpy as np
>>> import itertools
>>> a = np.array([1.0,1.0,2.0,2.0,2.0,2.0])
>>> [{'value': k, 'duration': len(list(v))} for k, v in itertools.groupby(a.tolist())]
[{'duration': 2, 'value': 1.0}, {'duration': 4, 'value': 2.0}]