Spring 批量申请 sqlcode -4499 sqlstate 08001

Question

Requirement:

Read from the file containing 100K records.

For each records, retrieve data from IBM DB2 database table and then retrieve data from NuoDB database table.

At last, insert the updated records in the NuoDB database table.

Design approached: Chunk-oriented processing where 1000 records will be read from the file and processed and written into the database.

Issue: After approx 75K records and running for almost 5 hours, the batch application failed with the below error:

Hibernate: select ... the SELECT query for DB2
Hibernate: select ... the SELECT query for NuoDB
2020-06-08 22:00:00.187  INFO [ ,,,] 32215 --- [       Thread-9] ConfigServletWebServerApplicationContext : Closing org.springframework.boot.web.servlet.context.AnnotationConfigServletWebServerApplicationContext@2a7f1f10: startup date [Mon Jun 08 17:22:51 BST 2020]; parent: org.springframework.context.annotation.AnnotationConfigApplicationContext@3972a855
2020-06-08 22:00:00.192  INFO [ ,,,] 32215 --- [       Thread-9] o.s.c.support.DefaultLifecycleProcessor  : Stopping beans in phase 0
2020-06-08 22:00:00.193  INFO [ ,,,] 32215 --- [       Thread-9] o.s.i.endpoint.EventDrivenConsumer       : Removing {logging-channel-adapter:_org.springframework.integration.errorLogger} as a subscriber to the 'errorChannel' channel
2020-06-08 22:00:00.193  INFO [ ,,,] 32215 --- [       Thread-9] o.s.i.channel.PublishSubscribeChannel    : Channel ' -1.errorChannel' has 0 subscriber(s).
2020-06-08 22:00:00.193  INFO [ ,,,] 32215 --- [       Thread-9] o.s.i.endpoint.EventDrivenConsumer       : stopped _org.springframework.integration.errorLogger
2020-06-08 22:00:00.195  INFO [ ,,,] 32215 --- [       Thread-9] o.s.s.c.ThreadPoolTaskScheduler          : Shutting down ExecutorService 'taskScheduler'
2020-06-08 22:00:00.196  INFO [ ,,,] 32215 --- [       Thread-9] o.s.jmx.export.MBeanExporter             : Unregistering JMX-exposed beans on shutdown
2020-06-08 22:00:00.203  INFO [ ,,,] 32215 --- [       Thread-9] j.LocalContainerEntityManagerFactoryBean : Closing JPA EntityManagerFactory for persistence unit 'default'
2020-06-08 22:00:00.203  INFO [ ,,,] 32215 --- [       Thread-9] j.LocalContainerEntityManagerFactoryBean : Closing JPA EntityManagerFactory for persistence unit 'default'
2020-06-08 22:00:00.203  INFO [ ,,,] 32215 --- [       Thread-9] j.LocalContainerEntityManagerFactoryBean : Closing JPA EntityManagerFactory for persistence unit 'default'
2020-06-08 22:00:00.203  INFO [ ,,,] 32215 --- [       Thread-9] j.LocalContainerEntityManagerFactoryBean : Closing JPA EntityManagerFactory for persistence unit 'default'
2020-06-08 22:00:00.203  INFO [ ,,,] 32215 --- [       Thread-9] com.zaxxer.hikari.HikariDataSource       : HikariPool-3 - Shutdown initiated...
2020-06-08 22:00:00.210  INFO [ ,,,] 32215 --- [       Thread-9] com.zaxxer.hikari.HikariDataSource       : HikariPool-3 - Shutdown completed.
2020-06-08 22:00:00.210  INFO [ ,,,] 32215 --- [       Thread-9] com.zaxxer.hikari.HikariDataSource       : HikariPool-2 - Shutdown initiated...
2020-06-08 22:00:00.211  INFO [ ,,,] 32215 --- [       Thread-9] com.zaxxer.hikari.HikariDataSource       : HikariPool-2 - Shutdown completed.
2020-06-08 22:00:00.212  INFO [ ,,,] 32215 --- [       Thread-9] com.zaxxer.hikari.HikariDataSource       : HikariPool-1 - Shutdown initiated...
2020-06-08 22:00:00.214  INFO [ ,,,] 32215 --- [       Thread-9] com.zaxxer.hikari.HikariDataSource       : HikariPool-1 - Shutdown completed.

What can be the actual cause of this issue? Is this like a database cant be triggered SELECT query for 100K times regularly for 4-6 hours.

I re-run the application with log-level in DEBUG mode and here is the error I got:

com.ibm.db2.jcc.am.DisconnectNonTransientConnectionException: [jcc][t4][2030][11211][4.19.72] A communication error occurred during operations on the connection's underlying socket, socket input stream, 
or socket output stream.  Error location: Command timeout check.  Message: Command timed out. ERRORCODE=-4499, SQLSTATE=08001

Answer 1

It seems like your database connection has been lost. You need to restart your job instance. If correctly configured, your job should restart from where it left off after the failure.