Pandas Groupby and cumsum with multiple conditions and columns - Python

Question

I have the following dataframe:

import pandas as pd

hits = {'id': ['A','A','A','A','A','A','B','B','B','C','C','C'],
        'datetime': ['2010-01-02 03:00:00','2010-01-02 03:00:14','2010-01-02 03:00:35','2010-01-02 03:00:38',
                    '2010-01-02 03:29:10','2010-01-02 03:29:35','2010-01-02 03:45:20','2010-01-02 06:10:05',
                    '2010-01-02 06:10:15','2010-01-02 07:40:15','2010-01-02 07:40:20','2010-01-02 07:40:25'],
        'uri_len': [10,20,25,15,20,10,20,25,15,30,40,45]
       }

df = pd.DataFrame(hits, columns = ['id', 'datetime','uri_len'])

df['datetime'] =  pd.to_datetime(df['datetime'], format='%Y-%m-%d %H:%M:%S')

print (df)

   id            datetime  uri_len
0   A 2010-01-02 03:00:00       10
1   A 2010-01-02 03:00:14       20
2   A 2010-01-02 03:00:35       25
3   A 2010-01-02 03:00:38       15
4   A 2010-01-02 03:29:10       20
5   A 2010-01-02 03:29:35       10
6   B 2010-01-02 03:45:20       20
7   B 2010-01-02 06:10:05       25
8   B 2010-01-02 06:10:15       15
9   C 2010-01-02 07:40:15       30
10  C 2010-01-02 07:40:20       40
11  C 2010-01-02 07:40:25       45

I want to group the hits by sessions, using id as the grouping by variable. For me, a session is an inactivity period of more than 15 seconds (calculated from the datetime column), or a decrease of the uri_len column, and in both cases comparing consecutive hits.

I know how to group by each condition individually:

df['session1'] = (df.groupby('id')['datetime']
               .transform(lambda x: x.diff().gt('15Sec').cumsum())
              )

df['session2'] = (df.groupby('id')['uri_len']
               .transform(lambda x: x.diff().lt(0).cumsum())
              ) 

Is there a way to combine both transformations in the same line, so the output is directly this?:

   id            datetime  uri_len  session
0   A 2010-01-02 03:00:00       10        0
1   A 2010-01-02 03:00:14       20        0
2   A 2010-01-02 03:00:35       25        1
3   A 2010-01-02 03:00:38       15        2
4   A 2010-01-02 03:29:10       20        3
5   A 2010-01-02 03:29:35       10        4
6   B 2010-01-02 03:45:20       20        0
7   B 2010-01-02 06:10:05       25        1
8   B 2010-01-02 06:10:15       15        2
9   C 2010-01-02 07:40:15       30        0
10  C 2010-01-02 07:40:20       40        0
11  C 2010-01-02 07:40:25       45        0

Answer 1

If I understood correctly, you want to add them?

df['session'] = df.groupby('id')['datetime'].transform(lambda x: 
x.diff().gt('15Sec').cumsum()) + df.groupby('id')['uri_len'].transform(lambda x: 
x.diff().lt(0).cumsum())

a more clear way:

s1 = df.groupby('id')['datetime'].transform(lambda x: 
x.diff().gt('15Sec').cumsum())

s2 = df.groupby('id')['uri_len'].transform(lambda x: x.diff().lt(0).cumsum())

df['session'] = s1+s2