Python Pandas datetime64 [ns]比较

Question

I'm trying to use indexing to select rows in my dataframe after the date 2011-01-01. I used following line of code to return only part of dataframe that is after 2011-01-01

  df = df[df.Date > np.datetime64('2011-01-01 00:00:00')]

I don't get an error. However, I only see dates that are of 2016 year nothing on 2011. When I manually open file I can see that there are plenty of entries starting with 2011 year.

What do I do wrong here? Any ideas?

Thanks!

Here is a sniped of data: enter image description here

Answer 1

After importing your data, it looks like all the values of the Date column are still there, even after filtering. It's just that your data is too large to be displayed fully on your console (take a look at pandas settings ). Therefore some of it is being (visually) truncated to fit the page.

The trick to use convert the Date column to a pandas datetime object and handle the filtering from there:

import pandas as pd

df = pd.read_csv('Crimes_-_2001_to_present.csv', header = 0)
df.Date = pd.to_datetime(df.Date)

filterer = df.Date > pd.to_datetime('2011-01-01 00:00:00')
df = df[filterer]

Now when you look at the 200th row in the Date column, you should get something:

df['Date'].iloc[200]
#Timestamp('2011-05-31 19:30:00')

At row 2000 in column Date :

df['Date'].iloc[2000]
#Timestamp('2013-09-19 20:45:00')

In essence, everything is there. Your console is perhaps too small to fit it all.

I hope this helps.