SQL request with composite primary key

Question

I have to get information from two databases. The one is an ORACLE and the other is a DB2. In my program (C#) I get in the first step the base informations of my objects from the ORACLE database. In the second step I want to add the information which are saved in the DB2. The table in the DB2 has composite primary key and I'm not sure which is the best way to request or if there is an alternative that I don't see at the moment.

For example: COLUMN1 and COLUMN2 are the composite primary key.

Variant 1:

 SELECT * 
   FROM (SELECT COLUMN1, COLUNN2, COLUMN3, ..., COLUMN1||'_'||COLUMN2 AS ID 
           FROM TABLE1) AS TEMP 
  WHERE ID='2011_123456' 
     OR ID='2011_987654'

Here I think the disadvantage is that for each row in the table the string concatenation is build and also the execution speed is comparatively slow because the primary key columns are indexed and the new one is not.

Variant 2:

SELECT COLUMN1, COLUMN2, COLUMN3, ..., COLUMN1||'_'||COLUMN2 AS ID 
  FROM TABLE1 
 WHERE (COLUMN1='2011' AND COLUMN2='123456') 
    OR (COLUMN1='2011' AND COLUMN2='987654')

This one is really fast but anytime I get an exception SQL0954C (Not enough storage is available in the application heap to process the statement).

Variant 3:

SELECT COLUMN1, COLUMN2, COLUMN3, ..., COLUMN1||'_'||COLUMN2 AS ID 
  FROM TABLE1 
 WHERE COLUMN1 IN ('2011') 
   AND COLUMN2 IN ('123456','987654')

This one is also slow in comparison to variant 2.

Some more numbers: TABLE1 has at the moment approx. 600k rows

I tried the variants and got the following execution times:
For 100 requested objects:
Variant 1: 3900ms
Variant 2: 218ms

For 400 requested objects:
Variant 1: 10983ms
Variant 2: 266ms

For 500 requested objects:
Variant 1: 12796ms
Variant 2: exception SQL0954C
Variant 3: 7061ms

Only looking on the times I would prefer variant 2 but there is the problem with the exception.

The databases are not under my control and I have only SELECT rights. What do you think is the best for this use case? Are there any other possibilities that I don't see?

Regards,
pkoeppe

Answer 1

Could you do a modification to variant 2 that

• defined a cursor
• bulk collected 100 rows (for example) into a pl/sql table
• do your processing
• fetch the next 100 rows

For example see http://oracletoday.blogspot.com/2005/11/bulk-collect_15.html

I had a problem very similar to this with Oracle and Informix.

Answer 2

Both variant 2 and 3 are sane. 1 is not sane.
As the computed column ID in 1 is not in any index the DB will be forced to do at least a full index scan. In 2 and 3 the DB can use indexes on both column1 and column2 to filter the result.

To find out whether 2 or 3 are best you need to study the execution plans for those queries.

One more note about indexes. Relevant indexes will be much more important than the difference between 2 and 3. Even if you only have select rights you can maybe suggest a composite index on (column1,column2) to the DBA if there are no such indexes already.

Edit
Another common approach when you have many values in WHERE COL IN (...) is to create a temp table (if you have permission) with all the values and join with that temp table instead. Sometimes you also need to create an index on the temp table to make it perform well.
In some DBMS:s you can use table valued parameters instead of temp tables, but I can not find anything like that for DB2.

Answer 3

SQL0954C can be resolved by tweaking your system's configuration. Have you explored that avenue yet? Find out more .

Answer 4

For variant 3, change

SELECT COLUMN1, COLUMN2, COLUMN3, ..., COLUMN1||'_'||COLUMN2 AS ID 
FROM TABLE1 
WHERE COLUMN1 IN ('2011') 
AND COLUMN2 IN ('123456','987654')

To:

SELECT COLUMN1, COLUMN2, COLUMN3, ..., COLUMN1||'_'||COLUMN2 AS ID 
FROM TABLE1 
WHERE COLUMN1 ='2011' 
AND COLUMN2 IN ('123456','987654')

If you're only searching for one value for COLUMN1 there's no reason to use IN.