Pandas DataFrame 在满足条件的前一行中查找最近的索引

Question

I have the following df1 dataframe:

    t       A
0   23:00   2
1   23:01   1
2   23:02   2
3   23:03   2
4   23:04   6
5   23:05   5
6   23:06   4
7   23:07   9
8   23:08   7
9   23:09   10
10  23:10   8

For each t (increments simplified here, not uniformly distributed in real life), I would like to find, if any, the most recent time tr within the previous 5 min where A(t)- A(tr) >= 4 . I want to get:

    t       A    tr
0   23:00   2
1   23:01   1
2   23:02   2
3   23:03   2
4   23:04   6    23:03
5   23:05   5    23:01
6   23:06   4
7   23:07   9    23:06
8   23:08   7
9   23:09   10   23:06
10  23:10   8    23:06

Currently, I can use shift(-1) to compare each row to the previous row like cond = df1['A'] >= df1['A'].shift(-1) + 4 .

How can I look further in time?

Answer 1

Assuming your data is continuous by the minute, then you can do usual shift:

df1['t'] = pd.to_timedelta(df1['t'].add(':00'))

df = pd.DataFrame({i:df1.A - df1.A.shift(i) >= 4 for i in range(1,5)})

df1['t'] - pd.to_timedelta('1min') * df.idxmax(axis=1).where(df.any(1))

Output:

0         NaT
1         NaT
2         NaT
3         NaT
4    23:03:00
5    23:01:00
6         NaT
7    23:06:00
8         NaT
9    23:06:00
10   23:06:00
dtype: timedelta64[ns]

Answer 2

I added a datetime index and used rolling() , which now includes time-window functionalities beyond simple index-window.

import pandas as pd
import numpy as np
import datetime

df1 = pd.DataFrame({'t' : [
        datetime.datetime(2020, 5, 17, 23, 0, 0),
        datetime.datetime(2020, 5, 17, 23, 0, 1),
        datetime.datetime(2020, 5, 17, 23, 0, 2),
        datetime.datetime(2020, 5, 17, 23, 0, 3),
        datetime.datetime(2020, 5, 17, 23, 0, 4),
        datetime.datetime(2020, 5, 17, 23, 0, 5),
        datetime.datetime(2020, 5, 17, 23, 0, 6),
        datetime.datetime(2020, 5, 17, 23, 0, 7),
        datetime.datetime(2020, 5, 17, 23, 0, 8),
        datetime.datetime(2020, 5, 17, 23, 0, 9),
        datetime.datetime(2020, 5, 17, 23, 0, 10)
        ], 'A' : [2,1,2,2,6,5,4,9,7,10,8]}, columns=['t', 'A'])
df1.index = df1['t']
df2 = df1
cond = df1['A'] >= df1.rolling('5s')['A'].apply(lambda x: x[0] + 4)
result = df1[cond]

Gives

t                         A
2020-05-17 23:00:04       6
2020-05-17 23:00:05       5
2020-05-17 23:00:07       9
2020-05-17 23:00:09      10
2020-05-17 23:00:10       8