Deleting records from Oracle 11g with JDBC

Question

I need to delete 10 000 records from a table containing 9 million records. The IDs which are to be deleted will be fetched from a complex query and stored in a Java collection.

I have 3 approaches to implement this

1) Create a prepared statement and add 10000 statements to the batch and execute it.

statement will look like this

Delete from <table_name> where id=?;

2) Write a 'in' query rather than using '=' in a batch. Like

Under this, the 10 000 IDs can be created as comma separated values in Java code and added to the query. Or, 10000 IDs are inserted in to a temporary table and make a select from that table in the sub query.

Delete from <table_name> where id in (<CSV>);
                 or
Delete from <table_name> where id in (select id from <temp_table>);

There are no constraints and indexes in the table. And I cannot add one, because I'm working on a existing table.

First option is taking ages to complete. It was running for 15hrs and still not completed.

Answer 1

You first version has a limit of 1000 values and tends to not perform well. The second approach may perform better but you have to have a global temporary table and populating it is an extra step.

You can convert your Java collection to an Oracle collection. You can create your own table type for this, but there are built-in ones like ODCINUMBERLIST which you can use here. You can the treat that as a table collection expression.

The details may vary slightly depending on your Java collection type, but the outline is something like:

ArrayDescriptor aDesc = ArrayDescriptor.createDescriptor("SYS.ODCINUMBERLIST",
  conn);
oracle.sql.ARRAY oraIDs = new oracle.sql.ARRAY(aDesc, conn, yourJavaCollectinOfIDs);

cStmt = (OracleCallableStatement) conn.prepareCall(
  "Delete from <table_name> 
   where id in (select column_value from table(?))");
cStmt.setArray(1, oraIDs);
cStmt.execute();

Unless it is already a simple array, You will need to convert your Java collection to an array in the call; eg if you're using an ArrayList called yourArrayList, you would do:

oracle.sql.ARRAY oraIDs = new oracle.sql.ARRAY(aDesc, conn, yourArrayList.toArray());

You will still suffer from the lack of a primary key or index but it will give Oracle a better chance to optimise it than the CSV list (or multiple CSV lists OR'd together as you have more than 1000 IDs).

Answer 2

You should not use the the first option by executing 10000 statements from your java code.

Creating a temp table is a good idea. But most of the time you can not have a IN (...) clause with more than 1000 items. So your approach with CSV may not success.

You may go for

Delete from <table_name> where id in (select id from <temp_table>);

but this way is not optimized either. It would be better to change your delete statement into this:

Delete from <table_name> m where exists (select id from <temp_table> t where m.id = t.id);

But if you do such operations frequently it's highly recommended to add some constraints and indexes to you <table_name> and even to your <temp_table> . It will boost your operations execution time like a charm.

Answer 3

The WHERE ... IN (...) is the way to go.

The IN clause can reference a temporary table that you've populated (your original idea), or it can contain any chosen (fixed) number of ? parameters. It will reduce the number of db roundtrips by a factor equal to the chosen number, but not necessarily to one. Iterate over your collection and process it in chunks.

Answer 4

Try like this.

Delete from <table_name> where
    id in (1, 2, 3, ... ,1000)
    or id in (1001, 1002, ... , 2000)
    ....