简体   繁体   中英

End pandas Grouper on max date in column

Minimal Reproducible Example

import numpy as np
import pandas as pd

np.random.seed(0)
dates = pd.date_range(start='1/1/2021', end='3/15/2021')
df = pd.DataFrame({'date': np.random.choice(dates, 1000), 
                   'label': np.random.choice(['a', 'b', 'c'], 1000)})

Result

    date    label
0   2021-02-14  a
1   2021-02-17  c
2   2021-03-06  a
3   2021-03-09  c
4   2021-03-09  b
... ... ...
995 2021-03-06  c
996 2021-01-14  b
997 2021-01-02  a
998 2021-01-03  c
999 2021-03-08  b

I am trying to group the date column by every 4 weeks starting with the last observed date (in this case, df['date'].max() gives '3/15/2021' , so I want the last date when grouping by date and label to be '3/15/2021' and for the other dates to be adjusted accordingly (28 days before 3/15, 56 days before 3/15, etc.).

However, I have not been able to do this with pd.Grouper . According to the docs , pd.Grouper takes an origin parameter that adjusts the grouping, but there is no option for basing it on the end date.

Is there a way to use pd.Grouper in a similar way to the following:

df.groupby([pd.Grouper(key='date', freq='28d', label='right'), 'label'])['label'].count()
date        label
2021-01-29  a        114
            b        135
            c        134
2021-02-26  a        125
            b        133
            c        123
2021-03-26  a         83
            b         81
            c         72
Name: label, dtype: int64

but instead have it set the last grouped by date end to be 3/15 (and have that last group contain all data from the 28 days since 3/15)?

We can try create the date count with div

df_sub = df.assign(v = ((df.date-df.date.max()).dt.days.sub(1)//28))
s = df_sub.groupby(['v','label']).agg({'label':'count'})
s = s.join(df_sub.groupby('v').date.max())
Out[41]: 
          label       date
v  label                  
-3 a         76 2021-01-18
   b         87 2021-01-18
   c         91 2021-01-18
-2 a        120 2021-02-15
   b        138 2021-02-15
   c        126 2021-02-15
-1 a        126 2021-03-15
   b        124 2021-03-15
   c        112 2021-03-15

Apparently pd.Grouper doesn't support negative frequent. I would resolve to grouping by Timedelta :

out = (df.groupby((max_date-df['date'])//pd.Timedelta('28d'))
   ['label'].value_counts()
)

# relabel the index
out.index = pd.MultiIndex.from_tuples([
    (max_date - pd.to_timedelta(x*28, unit='D'),y) for x,y in out.index
], names=['date','label'])

Output:

date        label
2021-03-15  a        126
            b        124
            c        112
2021-02-15  b        138
            c        126
            a        120
2021-01-18  c         91
            b         87
            a         76
Name: label, dtype: int64

The technical post webpages of this site follow the CC BY-SA 4.0 protocol. If you need to reprint, please indicate the site URL or the original address.Any question please contact:yoyou2525@163.com.

 
粤ICP备18138465号  © 2020-2024 STACKOOM.COM