How to loop over a large SQL result set in Java or get X number of rows at a time

Question

In my Spring Batch program I execute a SQL query, grab some information off the returned result set, then print it out to a file.

There are too many rows returned so the query has bad performance.

SELECT * 
FROM CIF
WHERE status > 1

This example query returns 500,000 rows.

I've tried selecting the top 5,000 rows at a time:

SELECT TOP 5000
FROM CIF
WHERE status > 1

The problem here is if I loop over this, I'll get the same 5,000 rows each time. I can't think of a way to loop and get 5,000 rows, print the info I need to a file, and grab the next 5,000 rows. Any ideas?

Answer 1

我不确定弹簧批次，但是到目前为止你使用sql你应该可以使用“limit”参数。

Answer 2

Spring Batch does this natively with our JdbcPagingItemReader . You can also stream records (instead of returning them all at once) with the JdbcCursorItemReader . The paging one handles generating the limit queries for you. You can read more about the JdbcPagingItemReader in the documentation here: https://docs.spring.io/spring-batch/4.0.x/reference/html/readersAndWriters.html#pagingItemReaders