spring batch with multiple datasources at runtime

Question

I have to crawl across some configured databases in a table . where each record specifies a schema to be read from . So we have to poll the table and run the job appropriately.

thought to use Spring batch(JdbcPagingItemReader) to read data from all the Schema's configured. if i have to configure this, How can i do it using Spring Batch ?

should i take multiple jobs with different reader for each database to read from, or is there any way where i can send the datasource at runtime for the Spring Batch to read data from ?

How can i manage Multiple databases for Single Spring Batch. If not is there any other suggestion for the Database Crawling(or Harvesting)?

Answer 1

If you are just running a query to get some data to then run other queries then this is not really something that aligns with what Spring Batch does. That just a standard JDBC or JPA type DAO/Service setup. You can use Quartz or Spring Scheduler to set a CRON value for when you what it to check the table(s).

Answer 2

There are 2 solutions using Spring Batch :

• Pass the DataSource properties (url, username, password....) at runtime through JobParameters . This means that the reading logic of the first read (the one which tells you what to read) has to be done outside the job
• Have a job with 2 steps, the first one doing the "metadata" read and storing result in the JobExecutionContext and the second one doing the actual read using previously stored values at runtime