Change data type of column from STRING format to DATE format

Question

I am reading a file from ADLS location, in that one column Period_Ending_Date is having data type as STRING. The Period_Ending_Date is having many dates in random order, I need to apply filter to get the latest date.

I'm trying this code:

select * from final_table WHERE Period_Ending_Date = (SELECT MAX(Period_Ending_Date) FROM final_table)

But the problem is I'm getting the day with maximum, not the latest date. I can understand this is happening because of STRING data type. Please guide me how I can change this column to DATE data type or any other alternative to get the solution of this.

I'm working with Scala and SQL on Azure Databricks.

Answer 1

what about changing SELECT MAX(Period_Ending_Date) FROM final_table to SELECT MAX(cast(Period_Ending_Date as date)) FROM final_table - performing explicit casting to date if date format is ISO8601 (YYYY-MM-DD) or using the to_date function ( doc ) to convert non-standard dates.