[英]Get rows whose timestamps are within specific sliding window time interval pandas (Time Series)
我有一個像這樣的 dataframe：
i = pd.to_datetime(np.random.randint(time.time(), time.time()+5000, 10), unit='ms').sort_values()
df = pd.DataFrame({'A':range(10),'B':range(10,30,2),'C':range(10,40,3)},index = i)
df
A B C
1970-01-19 04:28:30.030 0 10 10
1970-01-19 04:28:30.374 1 12 13
1970-01-19 04:28:31.055 2 14 16
1970-01-19 04:28:32.026 3 16 19
1970-01-19 04:28:32.234 4 18 22
1970-01-19 04:28:32.569 5 20 25
1970-01-19 04:28:32.595 6 22 28
1970-01-19 04:28:33.520 7 24 31
1970-01-19 04:28:33.882 8 26 34
1970-01-19 04:28:34.019 9 28 37
我想要的是，對於每個索引，在該索引的“1s”間隔內的最后一行：
df2
ix A B C
1970-01-19 04:28:30.030 1970-01-19 04:28:30.374 1 12 13
1970-01-19 04:28:30.374 1970-01-19 04:28:31.055 2 14 16
1970-01-19 04:28:31.055 1970-01-19 04:28:32.026 3 16 19
1970-01-19 04:28:32.026 1970-01-19 04:28:32.595 6 22 28
1970-01-19 04:28:32.234 1970-01-19 04:28:32.595 6 22 28
1970-01-19 04:28:32.569 1970-01-19 04:28:33.520 7 24 31
1970-01-19 04:28:32.595 1970-01-19 04:28:33.520 7 24 31
1970-01-19 04:28:33.520 1970-01-19 04:28:34.019 9 28 37
1970-01-19 04:28:33.882 1970-01-19 04:28:34.019 9 28 37
1970-01-19 04:28:34.019 nan nan nan nan
我目前正在使用循環執行此操作。 在每個索引處，我使用df.between_time來獲取時間間隔中的所有行，然后選擇最后一行。 但正如預期的那樣，它真的很慢。 我需要df.shift的時間，我檢查了tshift和shift(periods = 1, freq = 'S')但它們不像 shift 那樣工作，而是為每個索引添加指定的時間。 有人可以幫助我實現這一目標嗎？ 謝謝。
注意：所需 output 中的ix列是可選的。
PS：如果min_periods參數（如pd.df.rolling ）是可能的，那就太好了！
編輯：
對於起始df：
A B C
1970-01-19 04:28:34.883 0 10 10
1970-01-19 04:28:34.900 1 12 13
1970-01-19 04:28:35.531 2 14 16
1970-01-19 04:28:36.845 3 16 19
1970-01-19 04:28:37.664 4 18 22
1970-01-19 04:28:38.332 5 20 25
1970-01-19 04:28:38.444 6 22 28
1970-01-19 04:28:38.724 7 24 31
1970-01-19 04:28:38.787 8 26 34
1970-01-19 04:28:38.951 9 28 37
df['time'] = df.index
def last_time(time):
time = str(time)
start_time = datetime.datetime.strptime(time[11:],'%H:%M:%S.%f')
end_time = start_time + datetime.timedelta(0,1)
return df.between_time(start_time = str(start_time)[11:-7],end_time=
str(end_time)[11:-7]).iloc[-1]
df.apply(lambda x:last_time(x['time']),axis = 1)
# Output:
A B C time
1970-01-19 04:28:34.883 1 12 13 1970-01-19 04:28:34.900
1970-01-19 04:28:34.900 1 12 13 1970-01-19 04:28:34.900
1970-01-19 04:28:35.531 2 14 16 1970-01-19 04:28:35.531
1970-01-19 04:28:36.845 3 16 19 1970-01-19 04:28:36.845
1970-01-19 04:28:37.664 4 18 22 1970-01-19 04:28:37.664
1970-01-19 04:28:38.332 9 28 37 1970-01-19 04:28:38.951
1970-01-19 04:28:38.444 9 28 37 1970-01-19 04:28:38.951
1970-01-19 04:28:38.724 9 28 37 1970-01-19 04:28:38.951
但是正如您所看到的，我只能獲得second精度，即它正在考慮34 to 35之間，因此它缺少35.531 ，它在34.883和34.900的區間內。
假設您的時間已排序，那么第 2 行的相應行將嚴格大於第 1 行的行。例如：如果第 6 行是第 1 行的行，則第 2 行只需要搜索 >=6 的行
考慮到這一點，我們只需要遍歷索引一次（復雜度線性：O（n））：
import pandas as pd
from datetime import datetime
def time_compare(t1,t2):
return datetime.strptime(t1,'%Y-%m-%d %H:%M:%S.%f').timestamp() - datetime.strptime(t2,'%Y-%m-%d %H:%M:%S.%f').timestamp() < 1
index_j = []
cursor = 0
tmp = list(df.index)
for i in tmp:
if cursor < len(tmp):
pass
else:
index_j.append(cursor-1)
continue
while time_compare(tmp[cursor],i):
cursor += 1
if cursor < len(tmp):
pass
else:
break
index_j.append(cursor-1)
使用這個df：
>>> df
A B C
1970-01-19 04:28:34.883 0 10 10
1970-01-19 04:28:34.900 1 12 13
1970-01-19 04:28:35.531 2 14 16
1970-01-19 04:28:36.845 3 16 19
1970-01-19 04:28:37.664 4 18 22
1970-01-19 04:28:38.332 5 20 25
1970-01-19 04:28:38.444 6 22 28
1970-01-19 04:28:38.724 7 24 31
1970-01-19 04:28:38.787 8 26 34
1970-01-19 04:28:38.951 9 28 37
>>> index_j
[2, 2, 2, 4, 6, 9, 9, 9, 9, 9]
使用索引：
>>> [tmp[i] for i in index_j]
['1970-01-19 04:28:35.531', '1970-01-19 04:28:35.531', '1970-01-19 04:28:35.531', '1970-01-19 04:28:37.664', '1970-01-19 04:28:38.444', '1970-01-19 04:28:38.951', '1970-01-19 04:28:38.951', '1970-01-19 04:28:38.951', '1970-01-19 04:28:38.951', '1970-01-19 04:28:38.951']
我有點得到答案，因此分享，如果有人有更好的答案，歡迎您添加它。
i = pd.to_datetime(np.random.randint(time.time(), time.time()+5000, 10), unit='ms').sort_values()
df = pd.DataFrame({'A':range(10),'B':range(10,30,2),'C':range(10,40,3)},index = i)
df
df
A B C
1970-01-19 04:28:30.030 0 10 10
1970-01-19 04:28:30.374 1 12 13
1970-01-19 04:28:31.055 2 14 16
1970-01-19 04:28:32.026 3 16 19
1970-01-19 04:28:32.234 4 18 22
1970-01-19 04:28:32.569 5 20 25
1970-01-19 04:28:32.595 6 22 28
1970-01-19 04:28:33.520 7 24 31
1970-01-19 04:28:33.882 8 26 34
1970-01-19 04:28:34.019 9 28 37
df['time'] = df.index
def last_time(time):
time = str(time)
start_time = datetime.datetime.strptime(time[11:],'%H:%M:%S.%f')
end_time = start_time + datetime.timedelta(0,1)
tempdf = df.between_time(*pd.to_datetime([str(start_time),str(end_time)]).time).iloc[-1]
if str(tempdf['time']) == str(time):
tempdf.iloc[:] = np.nan
return tempdf
else:
return tempdf
df.apply(lambda x:last_time(x['time']),axis = 1)
A B C time
1970-01-19 04:28:34.883 2.0 14.0 16.0 1970-01-19 04:28:35.531000
1970-01-19 04:28:34.900 2.0 14.0 16.0 1970-01-19 04:28:35.531000
1970-01-19 04:28:35.531 NaN NaN NaN NaN
1970-01-19 04:28:36.845 4.0 18.0 22.0 1970-01-19 04:28:37.664000
1970-01-19 04:28:37.664 6.0 22.0 28.0 1970-01-19 04:28:38.444000
1970-01-19 04:28:38.332 9.0 28.0 37.0 1970-01-19 04:28:38.951000
1970-01-19 04:28:38.444 9.0 28.0 37.0 1970-01-19 04:28:38.951000
1970-01-19 04:28:38.724 9.0 28.0 37.0 1970-01-19 04:28:38.951000
1970-01-19 04:28:38.787 9.0 28.0 37.0 1970-01-19 04:28:38.951000
1970-01-19 04:28:38.951 NaN NaN NaN NaN
獲取時間戳在不規則時間間隔內的行熊貓（時間序列）
[英]Get rows whose timestamps are within irregular time intervals pandas (Time Series)
如何在 pandas 表的時間間隔內獲取具有 DatetimeIndex 的行？
[英]How to get rows having DatetimeIndex within time interval in pandas table?
以自定義間隔重新采樣 Pandas 時間序列並獲得一年內的間隔號
[英]Resample Pandas time series at custom interval and get interval number within a year
pandas：刪除另一系列時間索引的時間間隔內的所有行（即時間范圍排除）
[英]pandas: Remove all rows within time interval of another series's time index (i.e. time range exclusion)
聲明:本站的技術帖子網頁，遵循CC BY-SA 4.0協議，如果您需要轉載，請注明本站網址或者原文地址。任何問題請咨詢:yoyou2525@163.com.