获取具有匹配子记录的2个表之间的ID对

Question

I have 2 tables with the same structure.

FIELD 1      INT
FIELD 2      VARCHAR(32)   -- is a MD5 Hash

The query has to get matching FIELD 1 pairs from for records that have the exact combination of values for FIELD 2 in both TABLE 1 and TABLE 2.

These tables are pretty large ( 1 million records between the two ) but are deduced down to an ID and a Hash.

Example data:

1     A
1     B
2     A
2     D
2     E
3     G
3     H
4     E
4     D
4     C
5     E
5     D

8     A
8     B
9     E
9     D
9     C
10    F
11    G
11    H
12    B
12    D
13    A
13    B
14    E
14    A

The results of the query should be

\n8 1 \n9 4 \n11 3 \n13 1 \n

I have tried creating a concatenated string of FIELD 2 using a correlated sub-query and FOR XML PATH string trick I read on here but that is very slow.

Answer 1

You can try following query also -

SELECT t_2.Field_1, t_1.Field_1                          --1
  FROM table_1 t_1, table_2 t_2                          --2
 WHERE t_1.Field_2 = t_2.Field_2                         --3
 GROUP BY t_1.Field_1, t_2.Field_1                       --4
HAVING COUNT(*) = (SELECT COUNT(*)                       --5
                     FROM Table_1 t_1_1                  --6
                    WHERE t_1_1.Field_1 = t_1.Field_1)   --7
   AND COUNT(*) = (SELECT COUNT(*)                       --8
                     FROM Table_2 t_2_1                  --9
                    WHERE t_2_1.Field_1 =t_2.Field_1)    --10

Edit

First the requested set of result is the combination of Field1 from both the tables where respective Field2 is exactly same.

so for that you can use one method which I have posted above.

Here query will take the data from both the table based on field2 values (from line 1 to line 3) then it will group the data based on field1 from table1 and field1 from table2 (line 4)

till this step you will get the result having field1 from table1 and field2 from table2 where it exists (at least one) matching based on field2 from tables for respective field1 values.

after this you just need to filter the result for correct (exactly same values for field2 values for respective field1 column value). so that you can make condition on row count.

here my assumption is that you don't have multiple values for field1 and field2 combination in either tables

means following rows will not be present -

1 b 1 b

In any of the tables.

if so, the rows count got for table1 and table2 for same field2 values should be match with the rows present in table1 for field1 and same rows only should present in tables2 for field2 value.

for this condition query has condition on count(*) in having clause (from line 5 to line 10).

Answer 2

Let me try to explain this version of the query:

select t1.field1 as t1field1, t2.field1 as t2field1
from (select t1.*,
             count(*) over (partition by field1) as NumField2
      from table1 t1
     ) t1 full outer join
     (select t2.*,
             count(*) over (partition by field1) as NumField2
      from table2 t2
     ) t2
     on t1.field2 = t2.field2
where t1.NumField2 = t2.NumField2
group by t1.Field1, t2.Field1
having count(t1.field2) = max(t1.NumField2) and
       count(t2.field2) = max(t2.NumField2)

(which is here at SQLFiddle).

The idea is to compare the following counts for each pair of field1 values.

1. The number of field2 values on each.
2. The number of field2 values that they share.

All of these have to be equal.

Each subquery counts the number of values of field2 on each field1 value. For the first rows of your data, this produces:

1    A    2
1    B    2
2    A    3
2    D    3
2    E    3
. . .

And for the second table

8    A    2
8    B    2
9    E    3
9    D    3
9    C    3

Next, the full outer join is applied, requiring a match on both the count and the field2 value. This multiplies the data, producing rows such as:

1    A    2    8    A    2
1    B    2    8    B    2
2    A    3    NULL NULL NULL
2    D    3    9    D    3
2    E    3    9    E    3
NULL NULL NULL 9    C    3

And so on for all the possible combinations. Note that the NULL s appear due to the full outer join .

Note that when you have a pair, such as 1 and 8 that match, there are no rows with NULL values. When you have a pair with the same counts but they don't match, then you have NULL values. When you have a pair with different counts, they are filtered out by the where clause.

The filtering aggregation step applies these rules to get pairs that meet the first condition but not the second.

The having essentially removes any pair that has NULL values. When you count() a column, NULL values are not included. In that case, the count() on the column is fewer than the number of values expected ( NumField2 ).