使用准备好的语句将Spring批处理插入/更新到postgres

Question

I am trying to insert records to postgres table using prepared statements but I am getting following errors while using insert statements:

[main] ERROR org.springframework.batch.core.step.AbstractStep - Encountered an error executing step someStep in job someJob
org.springframework.dao.EmptyResultDataAccessException: Item 0 of 1 did not update any rows: [[com.abc.abc.abc.dao.orm.SomeORM@6653en,com.abc.abc.abc.dao.orm.SomeORM@75f04e]
   at org.springframework.batch.item.database.JdbcBatchItemWriter.write(JdbcBatchItemWriter.java:202)

My writer is :

package com.somepackage.writer;

public class trialClass implements ItemPreparedStatementSetter<List<someORM>> {

    @Override
    public void setValues(List<someORM> itemList,java.sql.PreparedStatement ps) throws SQLException,EmptyResultDataAccessException {
         try {
    for (ORM item : itemList) {
    for (int i = 0;i<itemList.size();i++)
        {
    ps.setInt(1, itemList.get(i).getPropOne());                         
    ps.setString(2, itemList.get(i).getPropTwo());
     ps.setInt(3, itemList.get(i).getPropThree());
     ps.setInt(4, itemList.get(i).getPropFour());
    ps.addBatch();
}
int[] rs = ps.executeBatch();
       } catch (Exception e) {
     e.printStackTrace();
    }}}

My xml file is something like this:

<beans:bean id="beanId" scope="step"
              class="org.springframework.batch.item.support.CompositeItemWriter">
              <beans:property name = "delegates">
                     <util:list>
                     <beans:ref bean= "someId"/>
                     <beans:ref bean= "someOtherId"/>
                     </util:list>
              </beans:property>
              </beans:bean>

<beans:bean id = "someId" scope = "step" 
              class="org.springframework.batch.item.database.JdbcBatchItemWriter">
              <beans:property name="dataSource" ref="someSource" />
              <beans:property name="sql" value = "${fetch.somequery.sql}" >  
              </beans:property>
              <beans:property name="itemPreparedStatementSetter" ref="someRef" />
       </beans:bean>

I am getting error only while inserting the records, updates are working fine (postgres table has primary key on first column) (In both the cases the records are getting inserted/updated in the postgres table but still getting this error in case of insert)

and I am using query as :

fetch.somequery.sql =INSERT INTO table1 (col1, col2, col3) SELECT ?,?,? WHERE NOT EXISTS (SELECT 1 FROM table1 WHERE col1= ?);

but when I am using plain insert query I am getting error as :

Caused by: org.postgresql.util.PSQLException: ERROR: duplicate key violates unique constraint "table1_pkey"  (seg576 sdw24:40000 pid=398877)
   at org.postgresql.core.v3.QueryExecutorImpl.receiveErrorResponse(QueryExecutorImpl.java:2062)

I have made sure that the records are not duplicate but still getting this error

Where am I doing it wrong? Can I use ORM to update data to postgres. If yes the how? Can I use jdbc template to update data to postgres. If yes the how?

How to handle this exception?

Answer 1

The entire record doesn't necessarily have to be a duplicate in order to violate a unique constraint . The only column(s) that need to be the same are the ones which comprise the constraint. Here, the error you provided says it's the table1_pkey constraint that's involved, so you would need to reference the DDL for that constraint to determine the columns.

From the code provided it's not possible to tell for sure, but I suspect what's happening is that two items in your list, for whichever property is referenced in table1_pkey , are the same.

It's generally best to use sequences for this sort of thing (assuming it's a single column), which Postgres will populate itself, as long as you just let it use the default value (and have the default value properly increment the sequence -- the serial type is syntactic sugar that does this automatically).

Also, when there's only one column involved, and it's meant to be the primary key, it can be declared as such inline, so something like this in the DDL : id serial primary key

If multiple columns are involved, the same general concept applies, but you wouldn't define the primary key quite the same way (it would defined later as a constraint, as it would be comprised of a column tuple), and you would need to make sure multiple sequences are used.

Response to comment from OP:

If the records are already present, trying to insert them again will definitionally violate the unique constraint. Are you mixing insert and update cases under a single insert operation? That will not work. Updates and inserts need to be separate. You can help guard against existence in an insert by adding a where exists sub-select in the insert, such that when it exists, the insert is a no-op. The updates would still need to occur separately.

The exception to that is the upsert , which is new to Postgres in 9.5 and fit what you're looking for here.

From the doc:

ON CONFLICT DO UPDATE guarantees an atomic INSERT or UPDATE outcome; provided there is no independent error, one of those two outcomes is guaranteed, even under high concurrency. This is also known as UPSERT — "UPDATE or INSERT".

Answer 2

I had a similar problem once and it's not caused by the writer bean.

The problem, in my case, was because I'm was using a non thread safe reader, so my source read (aleatory) some itens more than one time.

In your example sources I can see only the writers declarations.

I recomend you to check the reader and if you need a parallel reader, use the proper one or consider to use sequencial reader.