How to find duplicates in a SQL Server table which has trailing spaces values in a column

Question

select COL1, count(COL1)
from Table1
group by COL1
having count (COL1) > 1;

I have tried the above query and got some result based on data which do not have trailing spaces however the above query does not apply to data which has trailing spaces so I tried the below query and got no results. Please advice

select COL1, count(COL1)
from Table1
where COL1 in(select Ltrim(Rtrim(COL1))from Table1)
group by COL1
having count (COL1) > 1;

Answer 1

If you want to tally the text contents of COL1 ignoring leading and trailing whitespace, then just do that. Use ltrim(rtrim(COL1)) when aggregating:

select
    ltrim(rtrim(COL1)) AS COL1_trimmed
    count(*) cnt
from Table1
group by ltrim(rtrim(COL1))
having count(*) > 1;

Answer 2

In general, SQL Server ignores trailing spaces with varchar() . However, it does not when using char() . I am guessing the trailing "spaces" are not really spaces.

Here is an example .

with t as (
      select cast('a' as varchar(255)) as x union all
      select cast('a  ' as varchar(255))
    )
select t.x, count(*), min(t.x + '|') , max(t.x + '|')
from t
group by t.x;

This returns:

a   2   "a  |"  "a|"

(I added the double quotes to clarify the results.) Note that one row is returned, not two. But the spaces really are at the end of the values.

This leads me to suspect that the trailing characters are not spaces.

One way to investigate what they are is by using the ASCII() function.

Answer 3

Another way is to first remove the trailing and leading spaces from that column in your table.

If COL1 is a VARCHAR type:

update Table1
set COL1 = rtrim(ltrim(COL1))
where COL1 != rtrim(ltrim(COL1));

If COL1 is a CHAR type then you only need to left trim:

update Table1
set COL1 = ltrim(COL1)
where COL1 != ltrim(COL1);

After that cleanup, you can just use a grouping query without trimming the column

select COL1, count(*) as Total
from Table1
group by COL1
having count(*) > 1;