每當例如分類 pandas 時間序列更改 state 時如何提取時間戳

Question

我最近遇到了一個問題，即 pandas 時間序列包含一個可能需要多個狀態的信號，我對每個 state 的開始和結束時間戳感興趣，以便我可以為每個事件構建時隙。 輸入信號是帶有時間戳索引的 Pandas 系列，值可以是整數（例如類別的數字表示）或 NaN。 For NaN, I could assume that there had been no state change since the last state ( ffill would basically fix this) and that the state change happened exactly when it was logged (so the plot actually ought to be a step chart, not linearly interpolated如下圖所示）。

由於時隙是由它們的開始時間和結束時間定義的，因此我對一種可以提取圖底部所示時隙的(start time, end time)對的方法很感興趣。

數據：

import pandas as pd

data = [2,2,2,1,2,np.nan,np.nan,1,3,3,1,1,np.nan,
        2,1,np.nan,3,3,3,2,3,np.nan,3,1,2,1,3,3,1,
        np.nan,1,1,2,1,3,1,2,np.nan,2,1]
s = pd.Series(data=data, index=pd.date_range(start='1/1/2020', freq='S', periods=40))

Answer 1

好的，這就是我想出的方法。 如果有人有更有效或更優雅的方法，請分享。

import numpy as np
import pandas as pd

# Create the example Pandas Time Series
data = [2,2,2,1,2,np.nan,np.nan,1,3,3,1,1,np.nan,2,1,np.nan,3,3,3,2,3,np.nan,3,1,2,1,3,3,1,np.nan,1,1,2,1,3,1,2,np.nan,2,1]
dt = pd.date_range(start='1/1/2020', freq='S', periods=40)
s = pd.Series(data=data, index=dt)

# Drop NAN and calculate the state changes (not changing states returns 0)
s_diff = s.dropna().diff()

# Since 0 means no state change, remove them
s_diff = s_diff.replace(0,np.nan).dropna()

# Create a series that start with the time serie's initial condition, and then just the state change differences between the next states.
s_diff = pd.concat([s[:1], s_diff])

# We can now to a cumulative sum that starts on the initial value and adds the changes to find the actual states
s_states = s_diff.cumsum().astype(int)

# If the signal does not change in during the last timestamp, we need to ensure that we still get it.
s_states[s.index[-1]] = int(s[-1])

# Extract pairs of (start, end) timestamps for defining the timeslots. The .strftime method is only applied for readability. The following would probably be more useful:
# [(s_states.index[i], s_states.index[i+1] for i in range(len(s_states)-1)]
[(s_states.index[i].strftime('%M:%S'), s_states.index[i+1].strftime('%M:%S')) for i in range(len(s_states)-1)]
Out:
[('00:00', '00:03'),
 ('00:03', '00:04'),
 ('00:04', '00:07'),
 ('00:07', '00:08'),
 ('00:08', '00:10'),
 ('00:10', '00:13'),
 ('00:13', '00:14'),
 ('00:14', '00:16'),
 ('00:16', '00:19'),
 ('00:19', '00:20'),
 ('00:20', '00:23'),
 ('00:23', '00:24'),
 ('00:24', '00:25'),
 ('00:25', '00:26'),
 ('00:26', '00:28'),
 ('00:28', '00:32'),
 ('00:32', '00:33'),
 ('00:33', '00:34'),
 ('00:34', '00:35'),
 ('00:35', '00:36'),
 ('00:36', '00:39')]

Answer 2

這是一個稍微緊湊的方法。 我們將為每個組創建一個 label，然后使用groupby來確定該組的開始位置。 要形成這些組來ffill處理 NaN，請獲取差異並檢查不為 0 的位置（即它更改為任何狀態）。 此 Boolean 系列 forms 組的 cumsum。 由於下一組必須在上一組結束時開始，我們shift以獲取結束時間。

gps = s.ffill().diff().fillna(0).ne(0).cumsum()

df = s.reset_index().groupby(gps.to_numpy()).agg(start=('index', 'min'))
df['stop'] = df['start'].shift(-1)

Output

print(df.apply(lambda x: x.dt.strftime('%M:%S')))
## If you want a list of tuples:
# [tuple(zip(df['start'].dt.strftime('%M:%S'), df['stop'].dt.strftime('%M:%S')))]

    start   stop
0   00:00  00:03
1   00:03  00:04
2   00:04  00:07
3   00:07  00:08
4   00:08  00:10
5   00:10  00:13
6   00:13  00:14
7   00:14  00:16
8   00:16  00:19
9   00:19  00:20
10  00:20  00:23
11  00:23  00:24
12  00:24  00:25
13  00:25  00:26
14  00:26  00:28
15  00:28  00:32
16  00:32  00:33
17  00:33  00:34
18  00:34  00:35
19  00:35  00:36
20  00:36  00:39
21  00:39    NaT   # Drop the last row if you don't want this