使用条件和 groupby 操作 Pandas DataFrame

Question

DataFrame

I have the above DataFrame with hundreds of Instruments and different Dates. For each Instrument in every Date,

I wish to look for 2 rows of RecordType=='TRADE' (highlighted in yellow).

The first Trade type is right after the RecordType=='Control' , and

the second Trade type is 30 minutes after the first Trade, that is, the latest Trade within 30 minutes after the first trade.

In my example, 30 minutes after the first trade (9:59:47AM) should be 10:29:47AM, and if I convert the timestamp to Milliseconds (since midnight), it is 35987025 + (30mins * 60 * 1000) = 37787025. Therefore, the last Trade before 37787025 is at 37417668 Milliseconds, which is highlighted in yellow. Those Trades highlighted in red are NOT what I want.

May I know what is the best way to code this? I know I have to groupby(['Instrument', 'Date']) for the analysis. Thank you.

Answer 1

It's probably useful for you to know that you can use apply together with groupby . This is not a tested solution, but a rough guide how to get there:

def handle_single_group(df):
    trades = df[df.RecordType=='TRADE']
    first_trade = trades.iloc[0]

    latest_time_ok = first_trade['Time'] + timedelta(30)
    last_trade = trades[trades.Time <= latest_time_ok].iloc[-1]
    return pd.concat([first_trade, last_trade], axis=0)

df.groupby(['Instrument', 'Date']).apply(handle_single_group)

This assumes the trades are ordered by Time. You can also look into using argmax if that's not the case.