Cannot filter dates at pandas dataframe

Question

I have a .csv file which contains multiple columns and one of them is called Date and has date values of 2018 like that:

The format is Date at the .csv for this column.

I am going the following at my source code:

import pandas as pd

# Load data
data_daily = pd.read_csv('Desktop/data_daily.csv', keep_default_na=True)

# Filter data_daily down to only October
data_daily = data_daily[(data_daily['Date'] > '01/10/2018') & (data_daily['Date'] < '31/10/2018')]

# Save as a new .csv file
data_daily.to_csv('Desktop/final.csv', index=False)

However, the final .csv file has all the dates and not only the ones which I want.

I do not know if this makes a difference but keep in mind that there are multiple lines which have the same date.

How can I fix this?

Answer 1

First add parameter parse_dates in read_csv for parse column to datetimes:

data_daily = pd.read_csv('Desktop/data_daily.csv',
                         keep_default_na=True, 
                         parse_dates=['Date'],
                         dayfirst=True)

And then use your solution or alternative with between with converting strings to Timestamp :

s = pd.Timestamp('2018-10-01')
e = pd.Timestamp('2018-10-31')
data_daily = data_daily[data_daily['Date'].between(s, e, inclusive=False)]