熊猫数据框多重分组过滤

Question

I have the following dataframe:

df2 = pd.DataFrame({'season':[1,1,1,2,2,2,3,3],'value' : [-2, 3,1,5,8,6,7,5], 'test':[3,2,6,8,7,4,25,2],'test2':[4,5,7,8,9,10,11,12]},index=['2020', '2020', '2020','2020', '2020', '2021', '2021', '2021']) 
df2.index=  pd.to_datetime(df2.index)  
df2.index = df2.index.year
print(df2)

        season  test  test2  value
2020       1     3      4     -2
2020       1     2      5      3
2020       1     6      7      1
2020       2     8      8      5
2020       2     7      9      8
2021       2     4     10      6
2021       3    25     11      7
2021       3     2     12      5

I would like to filter it to obtain for each year and each season of that year the maximum value of the column 'value'. How can I do that efficiently?

Expected result:

print(df_result)

        season  value  test  test2
year                            
2020       1      3     2      5
2020       2      8     7      9
2021       2      6     4     10
2021       3      7     25    11

Thank you for your help,

Pierre

Answer 1

This is a groupby operation, but a little non-trivial, so posting as an answer.

(df2.set_index('season', append=True)
    .groupby(level=[0, 1])
    .value.max()
    .reset_index(level=1)
)
      season  value
2020       1      4
2020       2      8
2021       2      6
2021       3      7

Answer 2

You can elevate your index to a series, then perform a groupby operation on a list of columns:

df2['year'] = df2.index
df_result = df2.groupby(['year', 'season'])['value'].max().reset_index()

print(df_result)

   year  season  value
0  2020       1      4
1  2020       2      8
2  2021       2      6
3  2021       3      7

If you wish, you can make year your index again via df_result = df_result.set_index('year') .

To keep other columns use:

df2['year'] = df2.index
df2['value'] = df2.groupby(['year', 'season'])['value'].transform('max')

Then drop any duplicates via pd.DataFrame.drop_duplicates .

Update #1

For your new requirement, you need to apply an aggregation function for 2 series:

df2['year'] = df2.index

df_result = df2.groupby(['year', 'season'])\
               .agg({'value': 'max', 'test': 'last'})\
               .reset_index()

print(df_result)

   year  season  value  test
0  2020       1      4     6
1  2020       2      8     7
2  2021       2      6     2
3  2021       3      7     2

Update #2

For your finalised requirement:

df2['year'] = df2.index

df2['max_value'] = df2.groupby(['year', 'season'])['value'].transform('max')

df_result = df2.loc[df2['value'] == df2['max_value']]\
               .drop_duplicates(['year', 'season'])\
               .drop('max_value', 1)


print(df_result)

      season  value  test  test2  year
2020       1      3     2      5  2020
2020       2      8     7      9  2020
2021       2      6     4     10  2021
2021       3      7    25     11  2021

Answer 3

You can using get_level_values for bring index value into groupby

df2.groupby([df2.index.get_level_values(0),df2.season]).value.max().reset_index(level=1)
Out[38]: 
      season  value
2020       1      4
2020       2      8
2021       2      6
2021       3      7