[英]Finding similar sub-sequences in a time series?
I have thousands of time series (24 dimensional data -- 1 dimension for each hour of the day).我有数千个时间序列(24 维数据——一天中的每个小时都有 1 个维度)。 Out of these time series, I'm interested in a particular sub-sequence or pattern that looks like this:在这些时间序列中,我对如下所示的特定子序列或模式感兴趣:
I'm interested in sub-sequences that resemble the overall shape of the highlighted section -- that is, a sub-sequence with a sharp negative slope, followed by a period of several hours where the slope is relatively flat before finally ending with a sharp positive slope.我对类似于突出显示部分的整体形状的子序列感兴趣 - 即具有急剧负斜率的子序列,然后是几个小时的时间段,其中斜率相对平坦,然后最终以陡峭的正斜率。 I know the sub-sequences I'm interested in won't match each other exactly and most likely will be shifted in time, scaled differently, have longer/shorter periods where the slope is relatively flat, etc. but I would like to find a way to detect them all.我知道我感兴趣的子序列不会完全匹配,并且很可能会及时移动,缩放不同,有更长/更短的斜率相对平坦的周期等,但我想找到一种检测它们的方法。
To do this, I have developed a simple Heuristic (based on my definition of the highlighted section) to quickly find some of the sub-sequences of interest.为此,我开发了一个简单的启发式(基于我对突出显示部分的定义)来快速找到一些感兴趣的子序列。 However, I was wondering if there was a more elegant way (in Python) to search thousands of time series for the sub-sequence I'm interested in (while taking into account things mentioned above -- differences in time, scale, etc.)?但是,我想知道是否有一种更优雅的方式(在 Python 中)来搜索我感兴趣的子序列的数千个时间序列(同时考虑到上面提到的事情——时间、规模等方面的差异)。 )?
Edit : a year later I cannot believe how much I overcomplicated flatline and slope detection;编辑:一年后,我不敢相信我使平线和斜率检测变得多么复杂; stumbling on the same question, I realized it's as simple as偶然发现同一个问题,我意识到这很简单
idxs = np.where(x[1:] - x[:-1] == 0)
idxs = [i for idx in idxs for i in (idx, idx + 1)]
First line is implemented efficiently via np.diff(x)
;第一行通过np.diff(x)
有效实现; further, to eg detect slope > 5, use np.diff(x) > 5
.此外,例如检测斜率 > 5,请使用np.diff(x) > 5
。 The second line is since differencing tosses out right endpoints (eg diff([5,6,6,6,7]) = [1,0,0,1]
-> idxs=[1,2]
, excludes 3,
.第二行是因为差分抛出了正确的端点(例如diff([5,6,6,6,7]) = [1,0,0,1]
-> idxs=[1,2]
,不包括3,
。
Functions below should do;下面的功能应该做; code written with intuitive variable & method names, and should be self-explanatory with some readovers.用直观的变量和方法名称编写的代码,并且应该通过一些阅读来不言自明。 The code is efficient and scalable.该代码高效且可扩展。
Functionalities :功能:
Example :示例:
import numpy as np
import matplotlib.pyplot as plt
# Toy data
t = np.array([[ 5, 3, 3, 5, 3, 3, 3, 3, 3, 5, 5, 3, 3, 0, 4,
1, 1, -1, -1, 1, 1, 1, 1, -1, 1, 1, -1, 0, 3, 3,
5, 5, 3, 3, 3, 3, 3, 5, 7, 3, 3, 5]]).T
plt.plot(t)
plt.show()
# Get flatline indices
indices = get_flatline_indices(t, min_len=4, max_len=5)
plt.plot(t)
for idx in indices:
plt.plot(idx, t[idx], marker='o', color='r')
plt.show()
# Filter by edge slopes
lims_left = (-10, -2)
lims_right = (2, 10)
averaging_intervals = [1, 2, 3]
indices_filtered = filter_by_tail_slopes(indices, t, lims_left, lims_right,
averaging_intervals)
plt.plot(t)
for idx in indices_filtered:
plt.plot(idx, t[idx], marker='o', color='r')
plt.show()
def get_flatline_indices(sequence, min_len=2, max_len=6):
indices=[]
elem_idx = 0
max_elem_idx = len(sequence) - min_len
while elem_idx < max_elem_idx:
current_elem = sequence[elem_idx]
next_elem = sequence[elem_idx+1]
flatline_len = 0
if current_elem == next_elem:
while current_elem == next_elem:
flatline_len += 1
next_elem = sequence[elem_idx + flatline_len]
if flatline_len >= min_len:
if flatline_len > max_len:
flatline_len = max_len
trim_start = elem_idx
trim_end = trim_start + flatline_len
indices_to_append = [index for index in range(trim_start, trim_end)]
indices += indices_to_append
elem_idx += flatline_len
flatline_len = 0
else:
elem_idx += 1
return indices if not all([(entry == []) for entry in indices]) else []
def filter_by_tail_slopes(indices, data, lims_left, lims_right, averaging_intervals=1):
indices_filtered = []
indices_temp, tails_temp = [], []
got_left, got_right = False, False
for idx in indices:
slopes_left, slopes_right = _get_slopes(data, idx, averaging_intervals)
for tail_left, slope_left in enumerate(slopes_left):
if _valid_slope(slope_left, lims_left):
if got_left:
indices_temp = [] # discard prev if twice in a row
tails_temp = []
indices_temp.append(idx)
tails_temp.append(tail_left + 1)
got_left = True
if got_left:
for edge_right, slope_right in enumerate(slopes_right):
if _valid_slope(slope_right, lims_right):
if got_right:
indices_temp.pop(-1)
tails_temp.pop(-1)
indices_temp.append(idx)
tails_temp.append(edge_right + 1)
got_right = True
if got_left and got_right:
left_append = indices_temp[0] - tails_temp[0]
right_append = indices_temp[1] + tails_temp[1]
indices_filtered.append(_fill_range(left_append, right_append))
indices_temp = []
tails_temp = []
got_left, got_right = False, False
return indices_filtered
def _get_slopes(data, idx, averaging_intervals):
if type(averaging_intervals) == int:
averaging_intervals = [averaging_intervals]
slopes_left, slopes_right = [], []
for interval in averaging_intervals:
slopes_left += [(data[idx] - data[idx-interval]) / interval]
slopes_right += [(data[idx+interval] - data[idx]) / interval]
return slopes_left, slopes_right
def _valid_slope(slope, lims):
min_slope, max_slope = lims
return (slope >= min_slope) and (slope <= max_slope)
def _fill_range(_min, _max):
return [i for i in range(_min, _max + 1)]
声明:本站的技术帖子网页,遵循CC BY-SA 4.0协议,如果您需要转载,请注明本站网址或者原文地址。任何问题请咨询:yoyou2525@163.com.