[英]What is the fastest way to filter a pandas time series?
過濾 pandas 時間序列的最快方法是什么? 現在我使用 boolean 掩碼來過濾時間序列 ts:
import time
from datetime import datetime
import pandas as pd
import statistics
# create time series
idx = pd.date_range(start='2022-01-01', end='2023-01-01', freq="min")
ts = pd.Series(1, index=idx)
start_dt = datetime(2022, 1, 1, 0, 0, 0)
end_dt = datetime(2022, 1, 2, 0, 0, 0)
time_lst = []
# measure performance of boolean masking
for i in range(100):
start = time.time()
# 1st method
mask = (ts.index > start_dt) & (ts.index <= end_dt)
# 2nd method, nearly same velociy
# mask = np.where((ts.index > start_dt) & (ts.index <= end_dt), True, False)
time_lst.append(time.time() - start)
print(statistics.mean(time_lst))
filtered_ts = ts.loc[mask]
我想知道,如果這已經是最快的方法(這里每次運行 ~0.003 s)或者還有其他方法嗎? 我對不同的start_dt和end_dt使用了數千次掩碼,它總結了我想要減少的重要時間。
您的解決方案非常快:
%timeit ts[(ts.index > start_dt) & (ts.index <= end_dt)]
5.02 ms ± 413 µs per loop (mean ± std. dev. of 7 runs, 100 loops each)
%timeit ts[ts.index.to_series().between(start_dt, end_dt, inclusive='left')]
8.22 ms ± 160 µs per loop (mean ± std. dev. of 7 runs, 100 loops each)
但是,如果可能的話,使用Series.loc
包含兩個日期時間的更改解決方案會更快:
%timeit ts[(ts.index >= start_dt) & (ts.index <= end_dt)]
%timeit ts.loc[start_dt:end_dt]
138 µs ± 1.51 µs per loop (mean ± std. dev. of 7 runs, 10000 loops each)
聲明:本站的技術帖子網頁,遵循CC BY-SA 4.0協議,如果您需要轉載,請注明本站網址或者原文地址。任何問題請咨詢:yoyou2525@163.com.