Identifying overlapping events (datetime records) in a pandas dataframe

Question

I am having difficulty in trying to detect overlapping start_datetime and end_datetime in my dataset.

Currently my dataset looks like the following

but I'm trying to get to

Raw code to produce dataset

import pandas as pd
df = pd.DataFrame({
    'start_datetime':[
        '2000-01-01 02:23:49', '1997-12-20 07:22:10', '2000-01-05 03:42:29', '2002-02-25 17:20:09', '1999-06-30 03:33:20',
    ],
    'end_datetime':[
        '2000-01-06 04:50:20', '1998-12-20 01:24:12', '2000-03-01 11:01:11', '2003-02-25 22:05:02', '2000-01-01 02:50:30',
    ],
    
})
df['start_datetime'] = pd.to_datetime(df['start_datetime'])
df['end_datetime'] = pd.to_datetime(df['end_datetime'])
df

Is there a way (efficient or inefficient) to detect overlaps without sorting the columns?

Answer 1

Numpy broadcasting

s, e = df[['start_datetime', 'end_datetime']].to_numpy().T
m1 = (s[:, None] > s) & (s[:, None] < e) # Check if start time overlap
m2 = (e[:, None] < e) & (e[:, None] > s) # Check if ending time overlap

df['overlap'] = (m1 | m2).any(1)

Result

>>> df

       start_datetime        end_datetime  overlap
0 2000-01-01 02:23:49 2000-01-06 04:50:20     True
1 1997-12-20 07:22:10 1998-12-20 01:24:12    False
2 2000-01-05 03:42:29 2000-03-01 11:01:11     True
3 2002-02-25 17:20:09 2003-02-25 22:05:02    False
4 1999-06-30 03:33:20 2000-01-01 02:50:30     True