熊猫分组日期-特定日期

Question

I want to be able to group my data by user and then by specific date periods - to get counts and means and let them be created in new columns per date period.

My data looks something like:

df = pd.DataFrame({
"USER_ID": ["AA1", "AB1", "AA3", "CD3", "AB4", "AA1", "AA1", "AA3", "AB4", "AB4"],
"ACTIVITY_CATEGORY": [1, 2, 3, 4, 5, 6, 7, 8, 9, 10],
"DATE": ['2018-09-19', '2018-09-13', '2018-09-06', '2018-09-18', '2018-09-15', '2018-09-19', '2018-09-16', '2018-09-06', '2018-09-04', '2018-09-04']})

So, I usually do it as follows:

df.groupby(['USER_ID',pd.Grouper(key='DATE', freq='W')])['ACTIVITY_CATEGORY'].count()

But what I want now is to be able to get it for a specific week. Essentially being able to get something more like:

I have read the documentation on different manners of grouping in Grouper - and offsets that can be used. Still can not find something like this.

There is a fairly cumbersome way of doing this using a for loop and by using timedelta and subtracting 7 days from latest day - but it is highly inefficient on a large dataset. Looking for a more pythonic way.

Answer 1

is this similar to what you are trying to achieve

df['DATE'] = 'WEEK ' + pd.to_numeric(pd.to_datetime(df['DATE']).dt.day/7).apply(math.ceil).apply(str)
df.pivot_table(index=['USER_ID'],columns=['DATE'],aggfunc='count').fillna(0)

Out:

         ACTIVITY_CATEGORY
DATE    WEEK 1  WEEK 2  WEEK 3
USER_ID         
AA1     0.0     0.0     3.0
AA3     2.0     0.0     0.0
AB1     0.0     1.0     0.0
AB4     2.0     0.0     1.0
CD3     0.0     0.0     1.0

Answer 2

IIUC, you can try this:

df_new=df.groupby(['USER_ID',pd.Grouper(key='DATE', freq='W')])['ACTIVITY_CATEGORY']\
.count().reset_index()
df_new['week_num']=(df_new.DATE.dt.day//7)+1
print(df_new.pivot_table(index='USER_ID',columns=['week_num']).fillna(0))

            ACTIVITY_CATEGORY          
week_num                 2    3    4
USER_ID                             
AA1                    0.0  1.0  2.0
AA3                    2.0  0.0  0.0
AB1                    0.0  1.0  0.0
AB4                    2.0  1.0  0.0
CD3                    0.0  0.0  1.0

If week 1 is present, it should automatically populate.