在多个表中查找重复项

Question

I have a table (T1) and a table with attributes (T2). I'm looking to find records that have the same attributes as a record with provided id.

Here's the example. Given 1 I want to find 2 (ensuring that attributes match as well).

T1
ID | A | B
----------
1  | k | l
2  | k | l


T2
IDFK | C | D
-------------    
1    | w | x
1    | y | z
2    | w | x
2    | y | z

Here's the SQL I have so far:

SELECT * FROM T1 
JOIN T1 AS T1COPY ON T1.A = T1COPY.A, T1.B = T1COPY.B 
JOIN T2 ON T1.ID = T2.IDFK 
JOIN T2 AS T2COPY ON T1COPY.ID = T2COPY.IDFK 
   AND T2.C = T2COPY.C 
   AND T2.D = T2COPY.D
WHERE T1.ID = 1

but it's not working right as it's matching 2 even if attributes are different.

Answer 1

Here's the answer for MySQL: http://www.sqlfiddle.com/#!2/ec4fa/2

select h.* 
from 
(
    select x.*
    from t join t x using(a,b)
    where t.id = 1 and x.id <> 1  
) h
join 
(

    select coalesce(x.cpIdFk, x.uIdFk) as idFk  
    from
    (
      select cp.idFk as cpIdFk, u.idFk as uIdFk
      from 
      (
        select t.id as idFk, x.*
        from t cross join (select c, d from u where idFk = 1) as x
        where t.id <> 1      
      ) cp
      left join (select * from u where idFk <> 1) u using(idfk,c,d)

      union

      select cp.idFk,u.idFk
      from 
      (
        select t.id as idFk, x.*
        from t cross join (select c, d from u where idFk = 1) as x
        where t.id <> 1      
      ) cp
      right join (select * from u where idFk <> 1) u using(idfk,c,d)

    ) as x

    group by idFk
    having bit_and(cpidFk is not null and uIdFk is not null)

) d on d.idFk = h.id 
order by h.id;

Output for filter ID == 1:

| ID | A | B |
--------------
|  2 | k | l |
|  5 | k | l |

From these inputs:

CREATE TABLE t
    (ID int, A varchar(1), B varchar(1));

INSERT INTO t
    (ID, A, B)
VALUES
    (1, 'k', 'l'),
    (2, 'k', 'l'),
    (3, 'k', 'l'),
    (4, 'k', 'l'),
    (5, 'k', 'l'),
    (6, 'k', 'j');


CREATE TABLE u
    (IDFK int, C varchar(1), D varchar(1));

INSERT INTO u
    (IDFK, C, D)
VALUES
    (1, 'w', 'x'),
    (1, 'y', 'z'),

    (2, 'w', 'x'),
    (2, 'y', 'z'),

    (3, 'w', 'x'),
    (3, 'y', 'z'),
    (3, 'm', 'z'),

    (4, 'w', 'x'),

    (5, 'w', 'x'),
    (5, 'y', 'z'),

    (6, 'w', 'x'),
    (6, 'y', 'z');

Explanation here: Find duplicates across multiple tables

MySQL query look a little bit convoluted as it doesn't support FULL JOIN and it doesn't have CTEs too. We simulate FULL JOIN by unioning the result of LEFT JOIN and RIGHT JOIN

Answer 2

Second Revised Answer

Since the comments state that there can be duplicates rows in T2, a still more complex solution is needed. Here's a query that, I believe, generates the correct data.

-- Query 8B
SELECT x.id
  FROM (SELECT d2.id, d2.c, d2.d
          FROM (SELECT DISTINCT idfk AS id, c, d FROM t2 WHERE idfk != 1) AS d2
          JOIN (SELECT id
                  FROM (SELECT DISTINCT idfk AS id, c, d FROM t2 WHERE idfk != 1) AS x
                 GROUP BY id
                HAVING COUNT(*) = (SELECT COUNT(*)
                                     FROM (SELECT DISTINCT idfk AS id, c, d FROM t2 WHERE idfk  = 1) AS x
                                    GROUP BY id)
               ) AS j2
            ON j2.id = d2.id
       ) AS x
  JOIN (SELECT DISTINCT idfk AS id, c, d FROM t2 WHERE idfk  = 1) AS y
    ON x.c = y.c AND x.d = y.d
 GROUP BY x.id
HAVING COUNT(*) = (SELECT COUNT(*)
                     FROM (SELECT DISTINCT idfk AS id, c, d FROM t2 WHERE idfk  = 1) AS x
                    GROUP BY id);

I doubt if it is the simplest possible, but it is a logical continuation of the previous revised answer.

Example run

Here's the trace output of the query showing the steps as it was developed. The DBMS is IBM Informix Dynamic Server 11.70.FC2 running on Mac OS X 10.7.4, using SQLCMD v88.00 as the SQL command interpreter (no, not Microsoft's johnny-come-lately; the one I first wrote over twenty years ago).

+ BEGIN;
+ CREATE TABLE T1
(ID INTEGER NOT NULL PRIMARY KEY, a CHAR(1) NOT NULL, b CHAR(1) NOT  NULL);
+ INSERT INTO T1 VALUES(1, 'k', 'l');
+ INSERT INTO T1 VALUES(2, 'k', 'l');
+ INSERT INTO T1 VALUES(3, 'a', 'b');
+ INSERT INTO T1 VALUES(4, 'p', 'q');
+ INSERT INTO T1 VALUES(5, 't', 'v');
+ CREATE TABLE T2
(IDFK INTEGER NOT NULL REFERENCES T1, c CHAR(1) NOT NULL, d CHAR(1) NOT  NULL);
+ INSERT INTO T2 VALUES(1, 'w', 'x');
+ INSERT INTO T2 VALUES(1, 'y', 'z');
+ INSERT INTO T2 VALUES(2, 'w', 'x');
+ INSERT INTO T2 VALUES(2, 'w', 'x');
+ INSERT INTO T2 VALUES(2, 'y', 'z');
+ INSERT INTO T2 VALUES(3, 'w', 'x');
+ INSERT INTO T2 VALUES(3, 'y', 'b');
+ INSERT INTO T2 VALUES(3, 'y', 'z');
+ INSERT INTO T2 VALUES(4, 'w', 'x');
+ INSERT INTO T2 VALUES(5, 'w', 'x');
+ INSERT INTO T2 VALUES(5, 'y', 'z');
+ INSERT INTO T2 VALUES(5, 'w', 'x');
+ INSERT INTO T2 VALUES(5, 'y', 'z');
+ SELECT DISTINCT idfk AS id, c, d FROM t2 WHERE idfk != 1;
2|w|x
2|y|z
3|w|x
3|y|b
3|y|z
4|w|x
5|w|x
5|y|z
+ SELECT DISTINCT idfk AS id, c, d FROM t2 WHERE idfk  = 1;
1|w|x
1|y|z
+ SELECT id, COUNT(*) FROM (SELECT DISTINCT idfk AS id, c, d FROM t2 WHERE idfk != 1) AS x GROUP BY id;
2|2
5|2
3|3
4|1
+ SELECT id, COUNT(*) FROM (SELECT DISTINCT idfk AS id, c, d FROM t2 WHERE idfk  = 1) AS x GROUP BY id;
1|2
+ -- Query 5B - IDs having same count of distinct rows as ID = 1
SELECT id
  FROM (SELECT DISTINCT idfk AS id, c, d FROM t2 WHERE idfk != 1) AS x
 GROUP BY id
HAVING COUNT(*) = (SELECT COUNT(*)
                     FROM (SELECT DISTINCT idfk AS id, c, d FROM t2 WHERE idfk  = 1) AS x
                    GROUP BY id);
2
5
+ -- Query 6B
SELECT d2.id, d2.c, d2.d
  FROM (SELECT DISTINCT idfk AS id, c, d FROM t2 WHERE idfk != 1) AS d2
  JOIN (SELECT id
          FROM (SELECT DISTINCT idfk AS id, c, d FROM t2 WHERE idfk != 1) AS x
         GROUP BY id
        HAVING COUNT(*) = (SELECT COUNT(*)
                             FROM (SELECT DISTINCT idfk AS id, c, d FROM t2 WHERE idfk  = 1) AS x
                            GROUP BY id)
       ) AS j2
    ON j2.id = d2.id
 ORDER BY id;
2|w|x
2|y|z
5|w|x
5|y|z
+ -- Query 7B
SELECT x.id, y.id, x.c, y.c, x.d, y.d
  FROM (SELECT d2.id, d2.c, d2.d
          FROM (SELECT DISTINCT idfk AS id, c, d FROM t2 WHERE idfk != 1) AS d2
          JOIN (SELECT id
                  FROM (SELECT DISTINCT idfk AS id, c, d FROM t2 WHERE idfk != 1) AS x
                 GROUP BY id
                HAVING COUNT(*) = (SELECT COUNT(*)
                                     FROM (SELECT DISTINCT idfk AS id, c, d FROM t2 WHERE idfk  = 1) AS x
                                    GROUP BY id)
               ) AS j2
            ON j2.id = d2.id
       ) AS x
  JOIN (SELECT DISTINCT idfk AS id, c, d FROM t2 WHERE idfk  = 1) AS y
    ON x.c = y.c AND x.d = y.d
 ORDER BY x.id, y.id, x.c, x.d;
2|1|w|w|x|x
2|1|y|y|z|z
5|1|w|w|x|x
5|1|y|y|z|z
+ -- Query 8B
SELECT x.id
  FROM (SELECT d2.id, d2.c, d2.d
          FROM (SELECT DISTINCT idfk AS id, c, d FROM t2 WHERE idfk != 1) AS d2
          JOIN (SELECT id
                  FROM (SELECT DISTINCT idfk AS id, c, d FROM t2 WHERE idfk != 1) AS x
                 GROUP BY id
                HAVING COUNT(*) = (SELECT COUNT(*)
                                     FROM (SELECT DISTINCT idfk AS id, c, d FROM t2 WHERE idfk  = 1) AS x
                                    GROUP BY id)
               ) AS j2
            ON j2.id = d2.id
       ) AS x
  JOIN (SELECT DISTINCT idfk AS id, c, d FROM t2 WHERE idfk  = 1) AS y
    ON x.c = y.c AND x.d = y.d
 GROUP BY x.id
HAVING COUNT(*) = (SELECT COUNT(*)
                     FROM (SELECT DISTINCT idfk AS id, c, d FROM t2 WHERE idfk  = 1) AS x
                    GROUP BY id);
2
5
+ ROLLBACK;

First Revised Answer

Step 1: IDs having same count of rows as ID = 1

SELECT idfk AS id -- Query 5
  FROM t2
 WHERE idfk != 1
 GROUP BY idfk
HAVING COUNT(*) = (SELECT COUNT(*) FROM t2 WHERE t2.idfk = 1);

Step 2: Data corresponding Query 5

SELECT idfk AS id, c, d -- Query 6
  FROM t2
  JOIN (SELECT idfk AS id
          FROM t2
         WHERE idfk != 1
         GROUP BY idfk
        HAVING COUNT(*) = (SELECT COUNT(*) FROM t2 WHERE t2.idfk = 1)
       ) AS j2
    ON j2.id = t2.idfk
 ORDER BY id;

Step 3: Join rows from Query 6 with rows for ID = 1

SELECT x.id, y.id, x.c, y.c, x.d, y.d -- Query 7
  FROM (SELECT idfk AS id, c, d
          FROM t2
          JOIN (SELECT idfk AS id
                  FROM t2
                 WHERE idfk != 1
                 GROUP BY idfk
                HAVING COUNT(*) = (SELECT COUNT(*) FROM t2 WHERE t2.idfk = 1)
               ) AS j2
            ON j2.id = t2.idfk
       ) AS x
  JOIN (SELECT idfk AS id, c, d
          FROM t2 WHERE idfk = 1
       ) AS y
    ON x.c = y.c AND x.d = y.d
 ORDER BY x.id, y.id, x.c, x.d;

Step 4: IDs from Query 7 where the count is the same as the count for ID = 1

SELECT x.id
  FROM (SELECT idfk AS id, c, d
          FROM t2
          JOIN (SELECT idfk AS id
                  FROM t2
                 WHERE idfk != 1
                 GROUP BY idfk
                HAVING COUNT(*) = (SELECT COUNT(*) FROM t2 WHERE t2.idfk = 1)
               ) AS j2
            ON j2.id = t2.idfk
       ) AS x
  JOIN (SELECT idfk AS id, c, d
          FROM t2 WHERE idfk = 1
       ) AS y
    ON x.c = y.c AND x.d = y.d
 GROUP BY x.id
HAVING COUNT(*) = (SELECT COUNT(*) FROM t2 WHERE t2.idfk = 1);

Example run

The DBMS is IBM Informix Dynamic Server 11.70.FC2 running on Mac OS X 10.7.4, using SQLCMD v88.00 as the SQL command interpreter (no, not Microsoft's johnny-come-lately; the one I first wrote over twenty years ago).

+ BEGIN;
+ CREATE TABLE T1
(ID INTEGER NOT NULL PRIMARY KEY, a CHAR(1) NOT NULL, b CHAR(1) NOT  NULL);
+ INSERT INTO T1 VALUES(1, 'k', 'l');
+ INSERT INTO T1 VALUES(2, 'k', 'l');
+ INSERT INTO T1 VALUES(3, 'a', 'b');
+ INSERT INTO T1 VALUES(4, 'p', 'q');
+ CREATE TABLE T2
(IDFK INTEGER NOT NULL REFERENCES T1, c CHAR(1) NOT NULL, d CHAR(1) NOT  NULL);
+ INSERT INTO T2 VALUES(1, 'w', 'x');
+ INSERT INTO T2 VALUES(1, 'y', 'z');
+ INSERT INTO T2 VALUES(2, 'w', 'x');
+ INSERT INTO T2 VALUES(2, 'y', 'z');
+ INSERT INTO T2 VALUES(3, 'w', 'x');
+ INSERT INTO T2 VALUES(3, 'y', 'b');
+ INSERT INTO T2 VALUES(3, 'y', 'z');
+ INSERT INTO T2 VALUES(4, 'w', 'x');
+ SELECT t1.id AS id, t2.c, t2.d -- Query 1
  FROM t1
  JOIN t2 ON t1.id = t2.idfk;
1|w|x
1|y|z
2|w|x
2|y|z
3|w|x
3|y|b
3|y|z
4|w|x
+ -- Query 5 - IDs having same count of rows as ID = 1

SELECT idfk AS id
  FROM t2
 WHERE idfk != 1
 GROUP BY idfk
HAVING COUNT(*) = (SELECT COUNT(*) FROM t2 WHERE t2.idfk = 1);
2
+ SELECT idfk AS id, c, d
  FROM t2
  JOIN (SELECT idfk AS id
          FROM t2
         WHERE idfk != 1
         GROUP BY idfk
        HAVING COUNT(*) = (SELECT COUNT(*) FROM t2 WHERE t2.idfk = 1)
       ) AS j2
    ON j2.id = t2.idfk
 ORDER BY id;
2|w|x
2|y|z
+ SELECT x.id, y.id, x.c, y.c, x.d, y.d
  FROM (SELECT idfk AS id, c, d
          FROM t2
          JOIN (SELECT idfk AS id
                  FROM t2
                 WHERE idfk != 1
                 GROUP BY idfk
                HAVING COUNT(*) = (SELECT COUNT(*) FROM t2 WHERE t2.idfk = 1)
               ) AS j2
            ON j2.id = t2.idfk
       ) AS x
  JOIN (SELECT idfk AS id, c, d
          FROM t2 WHERE idfk = 1
       ) AS y
    ON x.c = y.c AND x.d = y.d
 ORDER BY x.id, y.id, x.c, x.d;
2|1|w|w|x|x
2|1|y|y|z|z
+ SELECT x.id
  FROM (SELECT idfk AS id, c, d
          FROM t2
          JOIN (SELECT idfk AS id
                  FROM t2
                 WHERE idfk != 1
                 GROUP BY idfk
                HAVING COUNT(*) = (SELECT COUNT(*) FROM t2 WHERE t2.idfk = 1)
               ) AS j2
            ON j2.id = t2.idfk
       ) AS x
  JOIN (SELECT idfk AS id, c, d
          FROM t2 WHERE idfk = 1
       ) AS y
    ON x.c = y.c AND x.d = y.d
 GROUP BY x.id
HAVING COUNT(*) = (SELECT COUNT(*) FROM t2 WHERE t2.idfk = 1);
2
+ ROLLBACK;

---

Original answer

This at least elicited sufficient clarification of the question.

As far as I can tell, if you have a sub-query like:

SELECT t1.id AS id, t2.c, t2.d  -- Query 1
  FROM t1
  JOIN t2 ON t1.id = t2.idfk

then you are looking for pairs of rows in the result set where the values in c and d are the same but the id values are different. So, we write the main query based on that:

SELECT j1.id, j2.id  -- Query 2
  FROM (SELECT t1.id AS id, t2.c, t2.d
          FROM t1
          JOIN t2 ON t1.id = t2.idfk
       ) AS j1
  JOIN (SELECT t1.id AS id, t2.c, t2.d
          FROM t1
          JOIN t2 ON t1.id = t2.idfk
       ) AS j2
    ON j1.c = j2.c AND j1.d = j2.d AND j1.id != j2.id

You can ensure you don't get both '1, 2' and '2, 1' by changing the != condition into either < or > .

If you want the rows that match a specific ID value in T1, then you can specify it in a WHERE clause:

SELECT j2.id  -- Query 3
  FROM (SELECT t1.id AS id, t2.c, t2.d
          FROM t1
          JOIN t2 ON t1.id = t2.idfk
       ) AS j1
  JOIN (SELECT t1.id AS id, t2.c, t2.d
          FROM t1
          JOIN t2 ON t1.id = t2.idfk
       ) AS j2
    ON j1.c = j2.c AND j1.d = j2.d AND j1.id != j2.id
 WHERE j1.id = 1;  -- 1 is the ID for which matches are sought

You can add conditions into the sub-queries if you wish (though a good optimizer might manage to do that for you):

SELECT j2.id  -- Query 4
  FROM (SELECT t1.id AS id, t2.c, t2.d
          FROM t1
          JOIN t2 ON t1.id = t2.idfk AND t1.id = 1
       ) AS j1
  JOIN (SELECT t1.id AS id, t2.c, t2.d
          FROM t1
          JOIN t2 ON t1.id = t2.idfk AND t1.id != 1
       ) AS j2
    ON j1.c = j2.c AND j1.d = j2.d
 WHERE j1.id = 1;  -- 1 is the ID for which matches are sought

The third condition in the main ON clause was redundant since, by construction, the ID values in the j1 sub-query are all 1 and the ID values in the j2 sub-query are all 'not 1'.

---

I fixed the issue with t2.id vs t2.idfk in the SQL, and I've run the 4 queries above. Each produces the answer I'd expect. There are two rows in the result set for, say, Query 4 because there are two pairs of rows in T1 such that both rows { 1, a , b } and { 2, a , b } exist in T2. If you only want the 2 two appear once, despite there being many matching rows, then you'll need to apply a DISTINCT to the SELECT.

In a comment, you say:

Unfortunately it will still return results even if one of the attributes does not match. How to match every single attribute in T2?

That requires an extended data set to demonstrate. When I added:

INSERT INTO T1 VALUES(3, 'a', 'b');
INSERT INTO T2 VALUES(3, 'a', 'z');
INSERT INTO T2 VALUES(3, 'y', 'b');

The value 3 only appeared in the results of Query 1, which is the only place it should appear.

Please illustrate what you are seeing as the incorrect behaviour, showing the sample data. I tested the queries above with the following SQL and the interleaved query results. The DBMS is IBM Informix Dynamic Server 11.70.FC2 running on Mac OS X 10.7.4, using SQLCMD v88.00 as the SQL command interpreter.

+ BEGIN;
+ CREATE TEMP TABLE T1
(ID INTEGER NOT NULL PRIMARY KEY, A CHAR(1) NOT NULL, B CHAR(1) NOT  NULL);
+ INSERT INTO T1 VALUES(1, 'k', 'l');
+ INSERT INTO T1 VALUES(2, 'k', 'l');
+ INSERT INTO T1 VALUES(3, 'a', 'b');
+ CREATE TEMP TABLE T2
(IDFK INTEGER NOT NULL, C CHAR(1) NOT NULL, D CHAR(1) NOT  NULL);
+ INSERT INTO T2 VALUES(1, 'w', 'x');
+ INSERT INTO T2 VALUES(1, 'y', 'z');
+ INSERT INTO T2 VALUES(2, 'w', 'x');
+ INSERT INTO T2 VALUES(2, 'y', 'z');
+ INSERT INTO T2 VALUES(3, 'a', 'z');
+ INSERT INTO T2 VALUES(3, 'y', 'b');
+ SELECT t1.id AS id, t2.c, t2.d -- Query 1
  FROM t1
  JOIN t2 ON t1.id = t2.idfk;
1|w|x
1|y|z
2|w|x
2|y|z
3|a|z
3|y|b
+ SELECT j1.id, j2.id -- Query 2
  FROM (SELECT t1.id AS id, t2.c, t2.d
          FROM t1
          JOIN t2 ON t1.id = t2.idfk
       ) AS j1
  JOIN (SELECT t1.id AS id, t2.c, t2.d
          FROM t1
          JOIN t2 ON t1.id = t2.idfk
       ) AS j2
    ON j1.c = j2.c AND j1.d = j2.d AND j1.id != j2.id;
1|2
1|2
2|1
2|1
+ SELECT j2.id -- Query 3
  FROM (SELECT t1.id AS id, t2.c, t2.d
          FROM t1
          JOIN t2 ON t1.id = t2.idfk
       ) AS j1
  JOIN (SELECT t1.id AS id, t2.c, t2.d
          FROM t1
          JOIN t2 ON t1.id = t2.idfk
       ) AS j2
    ON j1.c = j2.c AND j1.d = j2.d AND j1.id != j2.id
 WHERE j1.id = 1;
2
2
+ SELECT j2.id  -- Query 4
  FROM (SELECT t1.id AS id, t2.c, t2.d
          FROM t1
          JOIN t2 ON t1.id = t2.idfk AND t1.id = 1
       ) AS j1
  JOIN (SELECT t1.id AS id, t2.c, t2.d
          FROM t1
          JOIN t2 ON t1.id = t2.idfk AND t1.id != 1
       ) AS j2
    ON j1.c = j2.c AND j1.d = j2.d
 WHERE j1.id = 1;
2
2
+ ROLLBACK;

Answer 3

My approach to this is to combine the two columns of attributes into single values using group_concat. I can then easily find all ids that have the same attributes, and return these as the attributes.

select allts.id
from (select group_concat(c separator ';' order by c) as allcs,
             group_concat(d separator ';' order by d) as allds
      from t2
      where t2.id = 1
     ) t2_1 join
     (select t2.id, group_concat(c separator ';' order by c) as allcs,
             group_concat(d separator ';' order by d) as allds
      from t2
      group by t2.id
     ) allts
     on t2_1.allcs = allts.allcs and t2_1.allds = t2_1.allds join

This version does not take into account any information in t1. Your question only mentioned the attributes in t2.

Answer 4

Lets start by defining the attribute data which represents the input ID:

select idfk, c, d
from t2
where idfk = @ID

Now we can use this information to select potential matches existing in T2, where IDFK isnt @ID:

select x.idfk, x.c, x.d y.idfk as id2 from (
    select idfk, c, d
    from t2
    where idfk = @ID
) x left join t2 y on x.c = y.c and x.d = y.d
where y.idfk <> @ID 
  and y.idfk is not null

Data is the second query is a suitable match to data in the first query if the count of rows for each value of id2 is the same as the count of rows from the first query.

Hence:

select id2 from ( 
    select id2, count(*) as rowcount from (
        <second query>
    ) z
) rowsByID
where rowcount = (select count(*) from (<first query>) IDattributes)

I'm uncertain whether you intend that returned rows must match on A & B as well, or just on the data in Table 2, but if I assume they must match on A & B, then:

select ID from t1 
    join <third query> m on t1.id = m.id2
    join (select a, b from t1 where id = @id) prime_row on t1.a = prime_row.a and t1.b = prime_row.b

if you dont need A & B to match, drop the second join.

How's this?

Answer 5

This is probably going to be incredibly slow on a large table. (Edit: I now know that full joins are not available with mysql; but the first query is still valid for other systems and potentially a little easier to understand. Skip to the second one if you don't care about it.)

I used a question mark as the parameter marker. All should receive the same value of the "duplicate" id being matched. Add the condition and T.id <> ? to exclude the matching row from the result set. (I had thought OP wanted both rows 1 and 2.) tX represents the search space so it could also be excluded there and eliminated earlier in the process.

select *
from T1 as T
where T.id in (
    select coalesce(attrR.idfk, tX.id)
    from
        T1 as tX
        cross join
        (select * from T2 where T2.idfk = ?) as attrL
        full outer join T2 as attrR
            on      attrR.idfk = tX.id
                and attrR.c = attrL.c
                and attrR.d = attrL.d
    group by coalesce(attrR.idfk, tX.id)
    having count(*) =
        sum(case
                when attrR.c = attrL.c and attrR.d = attrL.d
                then 1 else 0
            end
        )
);

This gets around the lack of full outer join .

select *
from T1 as T
where T.id in (
    select attrR.idfk
    from
        T1 as tX
        cross join
        (select * from T2 where idfk = ?) as attrL
        right outer join
        T2 as attrR
            on      attrR.idfk = tX.id
                and attrR.c = attrL.c
                and attrR.d = attrL.d
        cross join
        (select count(*) as cnt from T2 where idfk = ?) as tC
    group by attrR.idfk
    having
        sum(case
                when attrR.c = attrL.c and attrR.d = attrL.d
                then 1 else 1000000
            end
        ) = min(tC.cnt)
);

This compound check is equivalent to the sum(case...) expression. One might feel better than the other.

    having
            count(attrL.idfk) = min(tC.cnt)
        and count(*) = min(tC.cnt)

The first and second query I provided do work, but only if each T1 has at least one attribute in T2. Here's a version that compensates by adding a dummy attribute to prevent empty sets in the intermediate results that mess it up. It's uglier so don't use it if not necessary for that case. (The full join version would need similar adjustments.)

select *
from T1 as T
where T.id in (
    select attrR.idfk
    from
        T1 as tX
        cross join
        (
            select c, d from T2 where idfk = ?
            union all
            select '!@#$%', '' -- add a dummy attribute
        ) as attrL
        right outer join
        (
            select idfk, c, d from T2
            union all
            select id, '!@#$%', '' from T1
        ) as attrR
            on      attrR.idfk = tX.id
                and attrR.c = attrL.c
                and attrR.d = attrL.d
        cross join
        (select count(*)+1 as cnt from T2 where idfk = ?) as tC -- note the +1
    group by attrR.idfk
    having
            count(tX.id) = min(tC.cnt)
        and count(*) = min(tC.cnt)
);

Answer 6

If by chance you will port your system to Postgresql, you can use FULL JOIN: http://www.sqlfiddle.com/#!1/1f0ef/1

with headers_matches as
(
    select x.*
    from t join t x using(a,b)
    where t.id = 1 and x.id <> 1
)
,cp as
(
    select t.id as idFk, x.*
    from t cross join (select c, d from u where idFk = 1) as x
    where t.id <> 1
)
,details_matches as
(
    select coalesce(cp.idFk,u.idFk) as idFk
    from cp
    full join (select * from u where idFk <> 1) u using(idfk,c,d)
    group by idFk
    having every(cp.idFk is not null and u.idFk is not null)
)
select h.* 
from headers_matches h
join details_matches d on d.idFk = h.id 
order by h.id;

Output for filter ID == 1:

| ID | A | B |
--------------
|  2 | k | l |
|  5 | k | l |

From these inputs:

CREATE TABLE t
    (ID int, A varchar(1), B varchar(1));

INSERT INTO t
    (ID, A, B)
VALUES
    (1, 'k', 'l'),
    (2, 'k', 'l'),
    (3, 'k', 'l'),
    (4, 'k', 'l'),
    (5, 'k', 'l'),
    (6, 'k', 'j');



CREATE TABLE u
    (IDFK int, C varchar(1), D varchar(1));

INSERT INTO u
    (IDFK, C, D)
VALUES
    (1, 'w', 'x'),
    (1, 'y', 'z'),

    (2, 'w', 'x'),
    (2, 'y', 'z'),

    (3, 'w', 'x'),
    (3, 'y', 'z'),
    (3, 'm', 'z'),

    (4, 'w', 'x'),

    (5, 'w', 'x'),
    (5, 'y', 'z'),

    (6, 'w', 'x'),
    (6, 'y', 'z');

---

How it works

We do the hardest part first, which is the details. We'll do header on latter part of this answer.

How it works, first we need to cross populate the details so we can do a proper full join on the details, so the gaps can be detected later:

with cp as -- cross populate
(
    select t.id as idFk, x.*
    from t cross join (select c, d from u where idFk = 1) as x
    where t.id <> 1
)
select *
from cp;

Output:

| IDFK | C | D |
----------------
|    2 | w | x |
|    2 | y | z |
|    3 | w | x |
|    3 | y | z |
|    4 | w | x |
|    4 | y | z |
|    5 | w | x |
|    5 | y | z |
|    6 | w | x |
|    6 | y | z |

Then from that cross-populated detail, we can do the proper FULL JOIN:

with cp as 
(
    select t.id as idFk, x.*
    from t cross join (select c, d from u where idFk = 1) as x
    where t.id <> 1
)
select 
    cp.idFk as cpIdFk, cp.c as cpC, cp.d as cpD,
    u.idFk as uFk, u.c as uC, u.d as Ud
from cp
full join (select * from u where idFk <> 1) u using(idfk,c,d);

Output:

| CPIDFK |    CPC |    CPD |    UFK |     UC |     UD |
-------------------------------------------------------
|      2 |      w |      x |      2 |      w |      x |
|      2 |      y |      z |      2 |      y |      z |
| (null) | (null) | (null) |      3 |      m |      z |
|      3 |      w |      x |      3 |      w |      x |
|      3 |      y |      z |      3 |      y |      z |
|      4 |      w |      x |      4 |      w |      x |
|      4 |      y |      z | (null) | (null) | (null) |
|      5 |      w |      x |      5 |      w |      x |
|      5 |      y |      z |      5 |      y |      z |
|      6 |      w |      x |      6 |      w |      x |
|      6 |      y |      z |      6 |      y |      z |

With that information at hand, we can now do the proper logic for detecting if there's a gap between the two sets, from the set above, we can see that those that have no gaps are #2, #5 and #6. For that we do this query:

with cp as
(
    select t.id as idFk, x.*
    from t cross join (select c, d from u where idFk = 1) as x
    where t.id <> 1
)
,details_matches as
(
    select coalesce(cp.idFk,u.idFk) as idFk
    from cp
    full join (select * from u where idFk <> 1) u using(idfk,c,d)
    group by idFk
    having every(cp.idFk is not null and u.idFk is not null)
)
select * from details_matches
order by idFk;

Output:

| IDFK |
--------
|    2 |
|    5 |
|    6 |

Then now we do the header comparision, which is easier:

with headers_matches as
(
    select x.*
    from t join t x using(a,b)
    where t.id = 1 and x.id <> 1
)
select * from headers_matches;

That should return header #2, #3, #4, #5 as they are identical to #1's header values:

Output:

| ID | A | B |
--------------
|  2 | k | l |
|  3 | k | l |
|  4 | k | l |
|  5 | k | l |

Finally, we combine the two queries:

with headers_matches as
(
    select x.*
    from t join t x using(a,b)
    where t.id = 1 and x.id <> 1
)
,cp as
(
    select t.id as idFk, x.*
    from t cross join (select c, d from u where idFk = 1) as x
    where t.id <> 1
)
,details_matches as
(
    select coalesce(cp.idFk,u.idFk) as idFk
    from cp
    full join (select * from u where idFk <> 1) u using(idfk,c,d)
    group by idFk
    having every(cp.idFk is not null and u.idFk is not null)
)
select h.* 
from headers_matches h
join details_matches d on d.idFk = h.id 
order by h.id;

Output:

| ID | A | B |
--------------
|  2 | k | l |
|  5 | k | l |

See the query progression here: http://www.sqlfiddle.com/#!1/1f0ef/1

I'll convert the Postgresql query to Mysql later.

UPDATE

Here's the MySQL version: Find duplicates across multiple tables

Answer 7

I've thought through your comments re my previous answer, and would propose a different approach.

select idfk, c, d from t2 where idfk = @ID 

this query identifies all the attribute-sets for @ID. Suppose we put this into a temporary table, then for EACH row in this table, identify all IDFKs in T2, where IDFK <> @ID, which match on all attribute values with the source row; Put all these rows into a new table.

My sql to do this would be: (you may need to adapt this for mysql)

create table #attribs (row# int, c, d);
insert #attribs (row#, c, d) values (0, null, null);

insert #attribs (row#, c, d)
select (select max row# from #attribs) + 1, c, d
from T2 where idfk = @ID;

delete #attribs where row# = 0;

create table #matchedattrib (idfk int)

while (select count(*) from #attribs) > 0 begin
    select @c = c, @d = d from #attribs where row# = (select min(row#) from #attribs);
    delete #attribs where row# = (select min(row#) from #attribs);

    insert #matchedattrib (idfk)
    select idfk from T2 where idfk <> @ID and T2.c = @c and T2.d = @d;
end

Having done this, any IDFK in this newer table, with the same number of rows as there are attribute sets for @ID (first query) has all the attributes of @ID.

select idfk, count(*) as tot_attribs
into #counts
from #matchedattrib
group by idfk
having count(*) = (select count(*) from (select idfk from T2 where idfk = @ID) x);

However, as you pointed out re my previous answer, these IDFKs could have other attributes as well, so then for IDFKs with the correct number of rows in the second table, you need to count that the rows existing for them in T2 is this same number - to verify that these matching attributes are in fact all the attributes for that IDFK - meaning a total match on attributes.

select idfk from #counts
where tot_attribs = (select count(*) from T2 where idfk = #counts.idfk)

If you also need to match on A + B, you'll have to fill that in yourself!