熊猫-根据Datetime列值删除DataFrame行

Question

I am currently writing a script where I want to drop some rows of my pandas dataframe according to Datetime values over several years (I want to drop rows where datetime is between February and May. So, I first tried the following code:

game_df['Date'] = game_df[(game_df['Date'].dt.month < 2) & (game_df['Date'].dt.month > 5)]

It gave me the same dataframe with NaN values in the 'Date' column over this period of time. So I tried the following code in order to drop the corresponding rows:

game_df['Date'] = game_df[(game_df['Date'].dt.month < 2) & (game_df['Date'].dt.month > 5)].drop(game_df.columns)

But it raised an error like: labels [u' Date ' u' other_column1 ' u' other_column2 ' u' other_column3 ' u' other_column4 '] not contained in axis

Does anyone can solve this problem?

Answer 1

I think you could try something like this using a list of Timestamp s:

If you want to exclude rows with specific dates:

game_df[~game_df['Date'].isin([pd.Timestamp('20150210'), pd.Timestamp('20150301')])]

The ~ is a not operator at the beginning of game_df in case you're not familiar with it. So it's saying to return the dataframe where the timestamps are not the two dates mentioned.

Edit: If you want to exclude a range of rows between specific dates:

game_df[~game_df['Date'].isin(pd.date_range(start='20150210', end='20150301'))]

Answer 2

Actually, I've found what I was looking for with the following code:

game_df = game_df[(game_df['Date'].dt.month != 2) & (game_df['Date'].dt.month != 3) & (game_df['Date'].dt.month != 4)\
                      & (game_df['Date'].dt.month != 5)]

It is pretty ugly and I truly think it can be done with a more efficient way but it works when it comes to exclude rows whose datetime values are located in a span of time.

Answer 3

Instead of dropping, I find query much more helpful. But you need to change arguments of course to include part of the data you want to keep.

df.query("Date.dt.month < 2 & Date.dt.month > 5", inplace=True)

if you want to use exact dates:

df.query("Date <= '2017-01-31' & Date >= '2017-05-01' ", inplace=True)