Mixed dates in column

Question

Working with Pandas: I have a very large dataframe where one column (imported from many.csv and.xlsx files) has mixed dates in different formats. I manage to unify all the separators /.- but still not able to unify the dates YYYY/MM/DD and DD/MM/YYYY into one of them (not really relevant which).

I have a column in a very large DataFrame:

Date
15-11-2021
2021-11-11
2021-11-20
19-11-2021
...

And I need:

Date
15-11-2021
11-11-2021
20-11-2021
19-11-2021
...

I tried solutions from old question no. 502726 and no. 49435438 but did not work

Like this:

datetime.datetime.strptime(df['Date'], "%d-%m-%Y").strftime("%Y-%m-%d")

But gave the error: AttributeError: type object 'datetime.datetime' has no attribute 'datetime'

Or the function definition:

new_date_format='%Y-%m-%d'
old_date_format='%d-%m-%Y'
def conv_dates_series(df, col, old_date_format, new_date_format):
    df[col] = pd.to_datetime(df[col], format=old_date_format).dt.strftime(new_date_format)
    return df

But quickly returns error when find a unexpected correct format is seen (probably a if/then clause will help here but I do not know how to use with to_datetime(format).

Any idea? Thanks!

Answer 1

Assuming the Date column values are strings (and assuming there aren't any weird formats like %Y-%d-%m ), I was able to simply use pd.Timestamp :

In [4]: df
Out[4]:
         Date
0  15-11-2021
1  2021-11-11
2  2021-11-20
3  19-11-2021

In [5]: df["Date"].apply(pd.Timestamp)
Out[5]:
0   2021-11-15
1   2021-11-11
2   2021-11-20
3   2021-11-19
Name: Date, dtype: datetime64[ns]

Alternatively, you can use pd.to_datetime(df["Date"]) .

Answer 2

You can simply use pd.to_datetime and let Pandas infers correct format:

>>> pd.to_datetime(df['Date'])
0   2021-11-15
1   2021-11-11
2   2021-11-20
3   2021-11-19
Name: Date, dtype: datetime64[ns]

Answer 3

Finally I found the solution, rather simple:

df['Date'] = pd.to_datetime(df['Date']).dt.strftime('%Y-%m-%d')