熊猫，Python：每5分钟计算一次唯一的名称方法出现次数

Question

I've been given a pandas dataframe in the following format:

             datetime                               name  mtd  code
0 2017-09-07 00:00:08                        profile/log  GET  300
1 2017-09-07 00:00:17                        profile/log  PUT  300
3 2017-09-07 00:00:19                             unknown PUT  200
4 2017-09-07 00:00:21                   extras/dashboard  GET  300
5 2017-09-07 00:00:23                       extras/stats  GET  300
6 2017-09-07 00:00:26                 extras/dashboard    GET  300
7 2017-09-07 00:00:29         extras/authz-profile/check  GET  200
8 2017-09-07 00:00:34                              about  PUT  300
9 2017-09-07 00:00:36                         extras/fav  GET  304
2 2017-09-07 00:00:44                       extras/store  GET  200

What I want to do is:

• to count number of occurrences for each name-mtd pair where response code starts with 3 for each 5 second interval starting from 2017-09-07 00:00:10 to 2017-09-07 00:00:40

The desirable output is:

     datetime_start     pair                      3??_count
2017-09-07 00:00:10     profile/log - GET         2
2017-09-07 00:00:15     -                         0
2017-09-07 00:00:20     extras/dashboard - GET    1
2017-09-07 00:00:20     extras/stats - GET        1
2017-09-07 00:00:25     extras/dashboard - GET    1
2017-09-07 00:00:30     about - PUT               1
2017-09-07 00:00:35     extras/fav - GET          1
2017-09-07 00:00:40     -                         0   

How am I to do that with pandas ?

I have written a piece of code that creates time periods as shown in the desirable output table, but don't know how to count a 3?? name-mtd pair for each 5-second period. I would highly appreciate any help!

data['datetime_start'] = pd.date_range(start="2017-09-07 00:00:10", end="2017-09-07 00:00:40", freq="5S")

Answer 1

create the start_date column

df['start_date']= df[' datetime'].apply(lambda dt: datetime.datetime(dt.year, dt.month, dt.day, dt.hour,dt.minute ,5*(dt.second//5)))

then you can aggregate

df.groupby(['start_date','name','mtd']).size()

Answer 2

here is one way to approach this

create a column that combines name-mtd as below

df['pair'] = df['name']+' - '+df['mtd']

then use PeriodIndex to specify the period to group the column datatime on as shown below

res = df.groupby([pd.PeriodIndex(df.datetime.dt.round('5s'),freq='5S'),
                'pair'])['pair'].count()

The output will be

datetime             pair                            
2017-09-07 00:00:10  profile/log - GET                   1
2017-09-07 00:00:15  profile/log - PUT                   1
2017-09-07 00:00:20  extras/dashboard - GET              1
                     unknown - PUT                       1
2017-09-07 00:00:25  extras/dashboard - GET              1
                     extras/stats - GET                  1
2017-09-07 00:00:30  extras/authz-profile/check - GET    1
2017-09-07 00:00:35  about - PUT                         1
                     extras/fav - GET                    1
2017-09-07 00:00:45  extras/store - GET                  1
Name: pair, dtype: int64