pandas compare timezone aware datetime field by minutes

Question

I have a datetime object like this:

df.iloc[:10]
0   2019-03-05 00:45:36.503277422+08:00
1   2019-03-05 00:46:36.404034571+08:00
2   2019-03-05 00:47:36.434888822+08:00
3   2019-03-05 00:48:36.535496247+08:00
4   2019-03-05 00:49:36.512082457+08:00
5   2019-03-05 00:50:36.515718466+08:00
6   2019-03-05 00:51:36.520325894+08:00
7   2019-03-05 00:52:36.523945647+08:00
8   2019-03-05 00:53:36.548567617+08:00
9   2019-03-05 00:54:36.740268213+08:00
Name: Date-Time, dtype: datetime64[ns, Asia/Shanghai]

I would to retrieve all rows if its time is later than 08:00:00 Asia/Shanghai time which means later than 00:00:00 UTC time. I have two questions:

1. how to write the condition in local time (Shanghai) instead of UTC time. Only df[df>'2019-03-05 00:00:00'] returns True . If I use df[df>'2019-03-05 08:00:00'] it will all be False .

2. how to use time only rather than have to prepend date before time. Instead of writing df[df>'2019-03-05 00:00:00'] , I want to write df[df>'00:00:00'] time only.

Many thanks!

Answer 1

You can add timezone information to scalar datetime and compare:

date = pd.to_datetime('2015-02-24').tz_localize('UTC').tz_convert('Asia/Shanghai')
print (date)
2015-02-24 08:00:00+08:00

Or:

date = pd.Timestamp('2015-02-24 08:00:00+08:00')

---

print (df[df > date])
0   2019-03-05 00:45:36.503277422+08:00
1   2019-03-05 00:46:36.404034571+08:00
2   2019-03-05 00:47:36.434888822+08:00
3   2019-03-05 00:48:36.535496247+08:00
4   2019-03-05 00:49:36.512082457+08:00
5   2019-03-05 00:50:36.515718466+08:00
6   2019-03-05 00:51:36.520325894+08:00
7   2019-03-05 00:52:36.523945647+08:00
8   2019-03-05 00:53:36.548567617+08:00
9   2019-03-05 00:54:36.740268213+08:00
Name: Date-Time, dtype: datetime64[ns, Asia/Shanghai]

And for second compare by time:

from datetime import time
print (df[df.dt.time > time(0,0,0)])
0   2019-03-05 00:45:36.503277422+08:00
1   2019-03-05 00:46:36.404034571+08:00
2   2019-03-05 00:47:36.434888822+08:00
3   2019-03-05 00:48:36.535496247+08:00
4   2019-03-05 00:49:36.512082457+08:00
5   2019-03-05 00:50:36.515718466+08:00
6   2019-03-05 00:51:36.520325894+08:00
7   2019-03-05 00:52:36.523945647+08:00
8   2019-03-05 00:53:36.548567617+08:00
9   2019-03-05 00:54:36.740268213+08:00
Name: Date-Time, dtype: datetime64[ns, Asia/Shanghai]

Or by timedeltas:

print (df[pd.to_timedelta(df.dt.strftime('%H:%M:%S')) > '00:00:00'])