删除 SQL 中 UNION 后的重复项

Question

I have two tables ( T1 and T2 )

-First, I select V1 , V2 , V3 , and V4 from T1 and remove duplicates based on V1 and V2 columns using the row_number() function.

-Second, I select V1 , V2 , V3 , and V4 from T2 and remove duplicates based on V1 and V2 columns using the row_number() function.

-Third, I used UNION to stack these two tables.

(WITH cte1 AS(
SELECT v1, v2, v3, v4,
row_number()over (PARTITION BY V1, V2  ORDER BY V1) rn
FROM T1)
SELECT V1, V2, V3, V4
FROM cte1 WHERE rn=1)
UNION
(WITH cte2 AS(
SELECT v1, v2, v3, v4,
row_number()over (PARTITION BY V1, V2  ORDER BY V1) rn
FROM T2)
SELECT V1, V2, V3, V4
FROM cte2 WHERE rn=1)

Now my question is: how can I remove duplicates from the final stacked table above using columns V1 , V2 , and V3 ?

NOTE: If there are duplicates in the final stacked table, then I need to remove the records where V4 =NULL. However, if no duplicates exist in the final stacked table, I still need to keep records where V4 =NULL.

Answer 1

You can use the same process of removing duplicates as you have used for both tables. It would look something like this:

WITH cteUnion AS
(   SELECT *, ROW_NUMBER() OVER (PARTITION BY V1,V2,V3 ORDER BY V1) AS rn
    FROM (
        (WITH cte1 AS(
        SELECT v1, v2, v3, v4,
        row_number()over (PARTITION BY V1, V2  ORDER BY V1) rn
        FROM T1)
        SELECT V1, V2, V3, V4
        FROM cte1 WHERE rn=1)
        UNION
        (WITH cte2 AS(
        SELECT v1, v2, v3, v4,
        row_number()over (PARTITION BY V1, V2  ORDER BY V1) rn
        FROM T2)
        SELECT V1, V2, V3, V4
        FROM cte2 WHERE rn=1)
    ) as union
)
SELECT *
FROM cteUnion
WHERE rn = 1

or you can use DISTINCT if you just want columns V1,V2,V3:

SELECT DISTINCT V1,V2,V3
FROM (
    (WITH cte1 AS(
    SELECT v1, v2, v3, v4,
    row_number()over (PARTITION BY V1, V2  ORDER BY V1) rn
    FROM T1)
    SELECT V1, V2, V3, V4
    FROM cte1 WHERE rn=1)
    UNION
    (WITH cte2 AS(
    SELECT v1, v2, v3, v4,
    row_number()over (PARTITION BY V1, V2  ORDER BY V1) rn
    FROM T2)
    SELECT V1, V2, V3, V4
    FROM cte2 WHERE rn=1)
) as union

Answer 2

UNION is the same as SELECT DISTINCT while UNION ALL is like using SELECT but for both tables.