Spring Batch: Multiple Item Readers in a single step

Question

I am a newbie in spring batch. The task I need to achieve in spring batch as follows:

1. Need to read some metadata from database.
2. Based on this metadata, I need to read some files.
3. After some processing, need to write those values from file to database.

My queries are the following:

a. For the 1st requirement, I needed to map the whole resultset to a single object, where Person related data are in 1 table and Pets related data are in another table and joined by person id.

public class PersonPetDetails {

    private String personName;
    private String personAddr;

    private int personAge;

    private List<Pet> pets;

For this I have written a custom Item reader which extends JdbcCursorItemReader.

public class CustomJDBCCusrorItemReader<T> extends JdbcCursorItemReader<T> {

    private ResultSetExtractor<T> resultSetExtractor;


    public void setResultSetExtractor(ResultSetExtractor<T> resultSetExtractor) {
        this.resultSetExtractor = resultSetExtractor;
    }

    @Override
    public void afterPropertiesSet() throws Exception {
        setVerifyCursorPosition(false);
        Assert.notNull(getDataSource(), "DataSource must be provided");
        Assert.notNull(getSql(), "The SQL query must be provided");
        Assert.notNull(resultSetExtractor, "ResultSetExtractor must be provided");
    }


    @Override
    protected T readCursor(ResultSet rs, int currentRow) throws SQLException {      
        return resultSetExtractor.extractData(rs);
    }
}

Is this the correct way to achieve my requirement? Or is there a much better way?

b.AFAIK, in spring batch there cannot be a step with just a reader, without a writer. Hence, I cannot call another set of reader in a different step of the Job. Then, how can I call multiple readers in a single step?

c. Also, based on some condition I may need to call a third set of Reader. How can I conditionally call a reader in a step?

Thanks for going through my post. I know it is long. Any help is much appreciated. Also, i guess an example code snippet would help me better understand the point. :)

Answer 1

I would recommend as below

High Level Design:

1. Partitioner It will deal with list of persons. Note: there is not Pet data pulled at this point of time.

2. Reader It will get a list of Pet which are belong to a Person. Note: Reader will return a list of Pet specific to a Person only.

3. Processor Base on a Pet-Person you will process base on your requirement.

4. Writer Based on your requirement to write to DB.

Low Level Code snippet:

1. Partitioner

    public class PetPersonPartitioner implements Partitioner { @Autowired private PersonDAO personDAO; @Override public Map<String, ExecutionContext> partition(int gridSize) { Map<String, ExecutionContext> queue = new HashMap<String, ExecutionContext>(); List<Person> personList = this.personDAO.getAllPersons(); for (Person person : personList) { ExecutionContext ec = new ExecutionContext(); ec.put("person", person); ec.put("personId", person.getId()); queue.put(person.getId(), ec); } return queue; } }

2. Reader

    <bean id="petByPersonIdRowMapper" class="yourpackage.PetByPersonIdRowMapper" /> <bean id="petByPesonIdStatementSetter" scope="step" class="org.springframework.batch.core.resource.ListPreparedStatementSetter"> <property name="parameters"> <list> <value>#{stepExecutionContext['personId']}</value> </list> </property> </bean>

 public class PetByPersonIdRowMapper implements RowMapper<PersonPetDetails> { @Override public BillingFeeConfigEntity mapRow(ResultSet rs, int rowNum) throws SQLException { PersonPetDetails record = new PersonPetDetails(); record.setPersonId(rs.getLong("personId")); record.setPetId(rs.getLong("petid"); ... ... }

3. Processor You can continue working on each PersonPetDetails object.