I want to get rid of part of a dataframe that has a certain date

Question

I'm trying to print a dataframe with datetimes corresponding to the 2/29/2020 date omitted in Jupyter. When I typed in the conditional statement on the top cell in the picture linked below and outputted the dataframe onto the bottom cell with all of the datetimes after 2/28/2020 22:00:00, only the dataframe row corresponding to just the first hour of the day (2/29/2020 00:00:00) was omitted and not the dataframe rows corresponding to the 2/29/2020 01:00:00 -> 2/29/2020 23:00:00 datetimes like I wanted. How can I change the conditional statement on the top cell which will make it so that all of the datetimes for 2/29/2020 will disappear?

Answer 1

Your question not clear. Lets assume I have the following;

Data

post_retrofit_without_2_29=pd.DataFrame({'Unit Datetime':['2020-02-28 23:00:00','2020-02-28 22:00:00','2020-02-29 22:00:00']})
print(post_retrofit_without_2_29)


 Unit Datetime
0  2020-02-28 23:00:00
1  2020-02-28 22:00:00
2  2020-02-29 22:00:00

Solution

To filter out by date, I have to coerce the datetime to date as follows;

post_retrofit_without_2_29['Unit Date']=pd.to_datetime(post_retrofit_without_2_29['Unit Datetime']).dt.strftime("%y-%m-%d")
print(post_retrofit_without_2_29)


  Unit Datetime     Unit Date
0  2020-02-28 23:00:00  20-02-28
1  2020-02-28 22:00:00  20-02-28
2  2020-02-29 22:00:00  20-02-29 

Filter

post_retrofit_without_2_29[post_retrofit_without_2_29['Unit Date']>'20-02-28']

 Unit Datetime Unit Date
2  2020-02-29 22:00:00  20-02-29

Answer 2

To omit all datetimes of 2/29/2020, you need to first convert the datetimes to dates in your comparison.

Change:

post_retrofit[post_retrofit['Unit Datetime'] != date(2020, 2, 29)]

To:

post_retrofit[post_retrofit['Unit Datetime'].dt.date != datetime(2020, 2, 29).date()]

Answer 3

You can do this easily by creating a pivot_table() with the dates as indexes. As a result of which, there will be no problem with the time.

post_retrofit_without_2_29_pivot = pd.pivot_table(data=post_retrofit_without_2_29 , index= post_retrofit_without_2_29['Unit Datetime'])

post_retrofit_without_2_29_pivot.loc[hourly_pivot.index != pd.to_datetime("2020-02-29") ]

I know this is a bit lengthy, but its simple to understand.

Hope, you got some help with this answer:}