如何从大型数据框中删除特定日期时间戳

Question

I have a big dataframe consisting of 600 days worth of data. Each day has 100 timestamps. I have a separate list of 30 days from which I want to data. How do I remove data from these 30 days from the dataframe? I tried a for loop, but it did not work. I know there is a simple method. But I don't know how to implement it.

df #is main dataframe which has many columns and rows. Index is a timestamp. 

df['dates'] = df.index.strftime('%Y-%m-%d') # date part of timestamp is sliced and  
#a new column is created. Instead of index, I want to use this column for comparing with bad list. 
bad_list # it is a list of bad dates   
for i in range(0,len(df)):
    for j in range(0,len(bad_list)):
        if str(df['dates'][i])== bad_list[j]:
            df.drop(df[i].index,inplace=True)

Answer 1

You can do the following

df['dates'] = df.index.strftime('%Y-%m-%d') 
#badlist should be in date format too. 

newdf = df[~df['dates'].isin(badlist)]
# the ~ is used to denote "not in" the list.

#if Jan 1, 2000 is a bad date, it should be in the list as datetime(2000,1,1)

Answer 2

You can perform simple comparison:

>>> dates = pd.Series(pd.to_datetime(np.random.randint(int(time()) - 60 * 60 * 24 * 5, int(time()), 12), unit='s'))
>>> dates
0    2019-03-19 05:25:32
1    2019-03-20 00:58:29
2    2019-03-19 01:03:36
3    2019-03-22 11:45:24
4    2019-03-19 08:14:29
5    2019-03-21 10:17:13
6    2019-03-18 09:09:15
7    2019-03-20 00:14:16
8    2019-03-21 19:47:02
9    2019-03-23 06:19:35
10   2019-03-23 05:42:34
11   2019-03-21 11:37:46

>>> start_date = pd.to_datetime('2019-03-20')
>>> end_date = pd.to_datetime('2019-03-22')
>>> dates[(dates > start_date) & (dates < end_date)]
1    2019-03-20 00:58:29
5    2019-03-21 10:17:13
7    2019-03-20 00:14:16
8    2019-03-21 19:47:02
11   2019-03-21 11:37:46

If your source Series is not in datetime format, then you will need to use pd.to_datetime to convert it.