How to clean/remove a column of Hidden characters or special characters in SQL Server?

Question

Issue: COMPLETE_DATE column is of type varchar (it is loaded like this, I cannot change this), I need to convert it to a Numeric datatype for manipulation. Format of date is yyyy-mm-dd .

When I run the convert function, I get this error:

Error converting data type varchar to numeric

Solution attempt 1: I ran the LEN function and noticed I was getting 11 returned instead of the expected 10. Initially I thought it was perhaps extra spaces so I

CONVERT(NUMERIC, RTRIM(LTRIM(COMPLETE_DATE)))

but it still errors out.

Solution attempt 2:

CONVERT(numeric, REPLACE(LTRIM(RTRIM([COMPLETE_DATE])), ' ', ''))

I am running a script from my predecessor where converts this column using the convert function but I am not sure at this point it is any extra space, or maybe special characters hiding, what else can I do to cleanse the column to just the string yyyy-mm-dd ?

UPDATE tbl SET ACADEMIC_YEAR = CASE WHEN CONVERT(NUMERIC,SUBSTRING(COMPLETE_DATE,1,4)+SUBSTRING(COMPLETE_DATE,6,2)+SUBSTRING(COMPLETE_DATE,9,2)) >= CONVERT(NUMERIC,SUBSTRING(COMPLETE_DATE,1,4)+'0701')

THEN CONVERT(NUMERIC,SUBSTRING(COMPLETE_DATE,1,4)) ELSE CONVERT(NUMERIC,SUBSTRING(COMPLETE_DATE,1,4))-1 END;

Answer 1

If the format is YYYY-MM-DD, then try:

try_convert(int, replace(academic_year, '-', ''))

I'm a little confused why you think that removing spaces will affect the conversion.

Answer 2

I found the only way to fix this issue, if you ever encounter it. It is not perfect, but it was an amalgamation of the responses answered by some of the users that replied. I went ahead and did a few things.

1. Because I was unable to identify whether it was an extra space of hidden character I went ahead and did a TRY_CONVERT function to a data while simultaneously using the RTRIM AND LTRIM but as suggested instead of a NUMERIC type I went with a DATE datatype, however the key really was to SUBSTRING'ing the actual date of 10. So it looked like this

try_convert(date,rtrim(ltrim(SUBSTRING(COMPLETE_DATE, 1, 10)))) as COMPLETE_DATE

this allowed for whatever extra space or character to be expunged and only 10 chars remained, then that string of 10 was CONVERTED to a DATE and the issue was resolved.

Not sure if this is the best way to do it, but it certaintly worked for me.