具有基于列总和的自定义回溯长度的熊猫滚动窗口

Question

Given a pandas dataframe with two columns, "atbats" and "hits", indexed by a date, is it possible to get the most recent historical batting average (average number of hits per atbat)? For example, the historical batting average could be that with the fewest atbats greater than 10. It is sort of like a rolling window with a conditional number of look-back periods. For example, given:

      date, atbats, hits, 
2017-01-01,      5,    2,
2017-01-02,      6,    3,
2017-01-03,      1,    1,
2017-01-04,      12,   3,
2017-01-04,      1,    0,

On the first day, there have been no historical atbats. On the second day, only 6. Since both are less than 10, they can be NaN or just 0.

On the third day, we would look back on the last two days and see 5+6 atbats with an average of (2+3)/(5+6) = 0.45 hits/atbat.

On the third day, we would look back on the last three days and get (2+3+1)/(5+6+1) = 0.5 hits/atbat.

On the fourth day, we would look back on just the last day and get 4/16 = 0.25 hits/atbat. Since the last day has more than 10 (16), we don't need to look any further.

The final dataframe would look like:

      date, atbats, hits,  pastAtbats, pastHits, avg,
2017-01-01,      5,    2,           0,       0,   0,
2017-01-02,      6,    3,           0,       0,   0,
2017-01-03,      1,    1,          11,       5,   0.45,
2017-01-04,      16,   4,          12,       6,   0.50,
2017-01-04,      1,    0,          16,       4,   0.25,

Is this sort of calculation possible in pandas?

The only solution I can think of is pure brute force - divide the hits by atbats in each row, replicate each row x times, where x = atbats, and then just do a rolling window of 10. But in my dataframe, the "atbats" average about 80 per day, so it would massively increase the size of the dataframe and total number of windows to calculate.

Answer 1

Use iterrows to achieve what you need. See below:

Original dataframe:

index atbats  hits
1       5     2
2       6     3
3       1     1
4      16     4
4       1     0
5       1     0
6      14     2
7       5     1

Code:

data = []
last = [0,0]
past_atbats = 0
past_hits = 0
for i, row in df.iterrows():
    if( last[0] >= 10):
        data.append(last.copy())
    else:
        data.append([0,0])

    if(row['atbats'] >= 10):
        last[0] = row['atbats']
        last[1] = row['hits']
    else:
        last[0] += row['atbats']
        last[1] += row['hits']

df_past = pd.DataFrame(data=data,index=df.index,columns=['past_atbats','past_hits'])
df = df.merge(df_past,left_index=True,right_index=True)
df['avg'] = df['past_hits'].divide(df['past_atbats'])

Result in:

index atbats  hits  past_atbats  past_hits       avg
1       5     2            0          0       NaN
2       6     3            0          0       NaN
3       1     1           11          5  0.454545
4      16     4           12          6  0.500000
4      16     4           16          4  0.250000
4       1     0           12          6  0.500000
4       1     0           16          4  0.250000
5       1     0           17          4  0.235294
6      14     2           18          4  0.222222
7       5     1           14          2  0.142857

Probably optimization can be done but I think it will helps you.