在 Pandas 的多列上使用 groupby 滚动最大值

Question

I am having a dataframe on which I want to compute maximum of rolling previous 3 month's value. Below is the dataframe:

VIN   Year_Month    Amount  
  V1    2012-01      196     
  V2    2012-01      113     
  V3    2012-01      177     
  V1    2012-02      154     
  V2    2012-02      129     
  V4    2012-02      156     
  V2    2012-03      100     
  V3    2012-03      174     
  V4    2012-03      127     
  V1    2012-04      139    
  V3    2012-04      194     
  V4    2012-04      178     

Using the following piece of code I am trying to compute the max value for each VIN in last 3 month

df['Max_3M_Value'] = df.groupby(['VIN','Year_Month'])['Amount'].rolling(3).max().fillna(0)

But the above code is giving error:

TypeError: incompatible index of inserted column with frame index

The resultant dataframe I am looking is :

  VIN  Year_Month   Amount  Max_3M_Value
   V1   2012-01      196     196
   V2   2012-01      113     129
   V3   2012-01      177     194
   V1   2012-02      154     196
   V2   2012-02      129     129
   V4   2012-02      156     178
   V2   2012-03      100     129
   V3   2012-03      174     194
   V4   2012-03      127     178
   V1   2012-04      139     196
   V3   2012-04      194     194
   V4   2012-04      178     178

I have to compute the max value using rolling only .I know we can solve this using pd.pivot_table() , but that is not needed.

What am I missing here.

Answer 1

You can avoid the error by using only the returned values, but you have to make sure that the order of the rows does not change during the operation:

df['Max_3M_Value'] = df.groupby(['VIN','Year_Month'])['Amount'].rolling(3).max().fillna(0).values

In your given example, your groups are too small (1 element each) to be able to apply a rolling with a window of 3:

>>> df.groupby(['VIN','Year_Month'])['Amount'].count()

VIN  Year_Month
V1   2012-01       1
     2012-02       1
     2012-04       1
V2   2012-01       1
     2012-02       1
     2012-03       1
V3   2012-01       1
     2012-03       1
     2012-04       1
V4   2012-02       1
     2012-03       1
     2012-04       1

I believe, what you really want is to group only by 'VIN':

df['Max_3M_Value'] = df.groupby('VIN')['Amount'].transform(lambda s: s.rolling(3, min_periods=1).max())

output:

   VIN Year_Month  Amount  Max_3M_Value
0   V1    2012-01     196           196
1   V2    2012-01     113           113
2   V3    2012-01     177           177
3   V1    2012-02     154           196
4   V2    2012-02     129           129
5   V4    2012-02     156           156
6   V2    2012-03     100           129
7   V3    2012-03     174           177
8   V4    2012-03     127           156
9   V1    2012-04     139           196
10  V3    2012-04     194           194
11  V4    2012-04     178           178

output for a rolling window of 2:

   VIN Year_Month  Amount  Max_3M_Value
0   V1    2012-01     196           196
1   V2    2012-01     113           113
2   V3    2012-01     177           177
3   V1    2012-02     154           196
4   V2    2012-02     129           129
5   V4    2012-02     156           156
6   V2    2012-03     100           129
7   V3    2012-03     174           177
8   V4    2012-03     127           156
9   V1    2012-04     139           154
10  V3    2012-04     194           194
11  V4    2012-04     178           178

If really what you want is a simple max per group, you can just do:

>>> df['Max_3M_Value'] = df.groupby('VIN')['Amount'].transform('max')
>>> df
   VIN Year_Month  Amount  Max_3M_Value
0   V1    2012-01     196           196
1   V2    2012-01     113           129
2   V3    2012-01     177           194
3   V1    2012-02     154           196
4   V2    2012-02     129           129
5   V4    2012-02     156           178
6   V2    2012-03     100           129
7   V3    2012-03     174           194
8   V4    2012-03     127           178
9   V1    2012-04     139           196
10  V3    2012-04     194           194
11  V4    2012-04     178           178

Answer 2

Try:

>>> df['Max_3M_Value'] = df.groupby('VIN')['Amount'] \
                           .rolling(3).max().bfill().astype(int) \
                           .reset_index(level=0, drop=True)

>>> df
   VIN Year_Month  Amount  Max_3M_Value
0   V1    2012-01     196           196
1   V2    2012-01     113           129
2   V3    2012-01     177           194
3   V1    2012-02     154           196
4   V2    2012-02     129           129
5   V4    2012-02     156           178
6   V2    2012-03     100           129
7   V3    2012-03     174           194
8   V4    2012-03     127           178
9   V1    2012-04     139           196
10  V3    2012-04     194           194
11  V4    2012-04     178           178