Retrieve large data over 200 MB from db JdbcTemplate

Question

Is there any better way to retrieve data in chunks instead of whole at a time and paginate them? I'm trying to retrieve large data over +300k rows (200 MB) using JdbcTemplate queryForList and return a paginated response for my API. I couldn't find any feasible column name to sort the data per my needs thus used ronum record_num. Below is my code.

final String sql = "SELECT * FROM (SELECT a.*, rownum record_num FROM (SELECT * FROM tableName WHERE MONTH = ? AND YEAR=?)a)"; 
List<Map<String, Object>> list = jdbcTemplate.queryForList(sql, month, year);

Answer 1

One way to do it would be to use rownum pseudocolumn and wrapping query. If you want to read rows 10-20 (do note rownum starts from 1):

SELECT * 
FROM ( SELECT *, rownum r FROM tableName WHERE month = ? AND year = ? ) 
WHERE r > 10 AND r <= 20

However if you are on Oracle 12c or newer you can use new OFFSET statement which database should optimize to skip reading redundant rows :

SELECT * 
FROM tableName 
WHERE month = ? AND year = ?
OFFSET 10 ROWS FETCH NEXT 10 ROWS ONLY

Answer 2

Since your question has been tagged with spring-boot , I am assuming your are using spring-boot in some capacity. Why don't you consider using spring data JPA?

<dependency>
            <groupId>org.springframework.boot</groupId>
            <artifactId>spring-boot-starter-data-jpa</artifactId>
</dependency>

This will allow you to use the Repository infrastructure. Then you would be able to use PagingAndSortingRepository . That will allow you to access your results in a paged way. Here's the reference material .