Check that any of three columns is within a column date range

Question

I have a DataFrame containing three datetime columns:

---

tp.loc[:, ['Arrival1', 'Arrival2', 'Departure']].head()

        Arrival1            Arrival2           Departure
0 2018-11-26 05:45:00 2018-11-26 12:00:00 2018-1-26 08:00:00
1 2018-11-26 22:00:00 2018-11-27 00:00:00 2018-11-26 23:00:00
2 2018-11-26 05:45:00 2018-11-26 08:15:00 2018-11-26 06:45:00
3 2018-11-26 07:30:00 2018-11-26 10:15:00 2018-11-26 08:30:00
4 2018-12-02 07:30:00 2018-12-02 21:30:00 2018-12-02 08:00:00

I want to get only the rows of tp whose Arrival 1, Arrival 2 or Departure (any of the three) are within the following column ranges (any of the rows):

db.loc[db['country'] == 'AT']

country        banStartDate          banEndDate
102      AT 2018-12-01 14:00:00 2018-12-01 22:59:00
161      AT 2018-12-01 23:00:00 2018-12-02 21:00:00
51       AT 2018-12-07 23:00:00 2018-12-08 22:59:00

---

In this example, I want only row #4 to be retrieved from tp since Arrival2 is within the date range of db.

Is there an easy way to do so?

Answer 1

After reading in your dataframes with pd.read_csv() , you can use pd.concat() with a boolean mask and list comprehension, followed by drop_duplicates() :

from io import StringIO
import pandas as pd

df1 = StringIO('''
            Arrival1            Arrival2           Departure
0  2018-11-26 05:45:00  2018-11-26 12:00:00  2018-1-26 08:00:00
1  2018-11-26 22:00:00  2018-11-27 00:00:00  2018-11-26 23:00:00
2  2018-11-26 05:45:00  2018-11-26 08:15:00  2018-11-26 06:45:00
3  2018-11-26 07:30:00  2018-11-26 10:15:00  2018-11-26 08:30:00
4  2018-12-02 07:30:00  2018-12-02 21:30:00  2018-12-02 08:00:00
''')

df2 = StringIO('''
    country        banStartDate          banEndDate
102      AT  2018-12-01 14:00:00  2018-12-01 22:59:00
161      AT  2018-12-01 23:00:00  2018-12-02 21:00:00
51       AT  2018-12-07 23:00:00  2018-12-08 22:59:00
''')

tp = pd.read_csv(df1, sep=r'\s{2,}', engine='python', parse_dates=[0,1,2])
db = pd.read_csv(df2, sep=r'\s{2,}', engine='python', parse_dates=[1,2]).reset_index()

pd.concat([tp.loc[((tp>db.loc[i,'banStartDate']) & (tp<db.loc[i,'banEndDate'])).any(axis=1)] for i in range(db.shape[0])]).drop_duplicates()

Returns:

             Arrival1            Arrival2           Departure
4 2018-12-02 07:30:00 2018-12-02 21:30:00 2018-12-02 08:00:00

Answer 2

You can use the pandas.DataFrame.any with axis = 'row'(or 1) to find where the dates are between start and end. You will need 3 of these or a for loop for however many 'country' column of db there are.

Also, I believe(I could be wrong) you will need to convert those strings into python datetime variables. The code would look similar to this;

tp[(datetime.strptime(Start_Date, '%Y-%d-%m %H:%M:%S')> tp >datetime.strptime(End_Date, '%Y-%d-%m %H:%M:%S')).any(axis=1)]