I am working on Spring Batch job that moves data from Sql Server to Cassandra. I am using Spring Data JPA to read and write the data. I have created entities and JPA Repositories for both databases.
Now I can not figure out how to use my JpaRepostorty with Spring Batch ItemReader. I have searched on internet and found few refs where they mentioned to use JpaPageItemReader. But that requires specifying query and configuring other details. But I cannot figure out how to use existing JpaRepository that I have. Below is snippet of relevant code-
My JpaRepostory for SQL Server -
public interface ScanJpaRepository extends JpaRepository<Scan, Integer>
{
@Transactional(readOnly = true)
@Query("select s from Scan s left join fetch s.projectVersion")
Stream<Scan> findAllScan();
}
My Spring Batch Job -
@Configuration
@EnableBatchProcessing
public class SSCBatchConfigurationCassandra {
@Autowired
public JobBuilderFactory jobBuilderFactory;
@Autowired
public StepBuilderFactory stepBuilderFactory;
@Bean
public PlatformTransactionManager transactionManager() {
return new ResourcelessTransactionManager();
}
@Bean
public JobExplorer jobExplorer() throws Exception {
MapJobExplorerFactoryBean jobExplorerFactory = new MapJobExplorerFactoryBean(mapJobRepositoryFactoryBean());
jobExplorerFactory.afterPropertiesSet();
return jobExplorerFactory.getObject();
}
@Bean
public MapJobRepositoryFactoryBean mapJobRepositoryFactoryBean() {
MapJobRepositoryFactoryBean mapJobRepositoryFactoryBean = new MapJobRepositoryFactoryBean();
mapJobRepositoryFactoryBean.setTransactionManager(transactionManager());
return mapJobRepositoryFactoryBean;
}
@Bean
public JobRepository jobRepository() throws Exception {
return mapJobRepositoryFactoryBean().getObject();
}
@Bean
public JobLauncher jobLauncher() throws Exception {
SimpleJobLauncher simpleJobLauncher = new SimpleJobLauncher();
simpleJobLauncher.setJobRepository(jobRepository());
return simpleJobLauncher;
}
@Bean
public ItemReader<Project> reader() {
**// how to read from ScanJpaRepository ??**
}
@Bean
public CassandraItemProcessor processor() {
return new CassandraItemProcessor();
}
@Bean
public ItemWriter<CassandraProject> cqlWriter() {
final CassandraBatchItemWriter writer = new CassandraBatchItemWriter();
return writer;
}
// tag::jobstep[]
@Bean
public Job importUserJob(JobCompletionNotificationListener listener, Step step1) {
return jobBuilderFactory.get("importUserJob")
.incrementer(new RunIdIncrementer())
.listener(listener)
.flow(step1)
.end()
.build();
}
@Bean
public Step step1() {
return stepBuilderFactory.get("step1")
.<Project, CassandraProject> chunk(100)
.reader(reader())
.processor(processor())
.writer()
.build();
}
// end::jobstep[]
}
Update #1: I added item reader to my batch job as mentioned.
@Configuration
@EnableBatchProcessing
public class FortifySSCBatchConfigurationCassandra {
....
@Autowired
public ScanItemReader itemReader;
.....
@Bean
public Step step1() {
return stepBuilderFactory.get("step1")
.<Scan, CScan> chunk(100)
.reader(itemReader)
.processor(processor())
.writer(cqlWriter())
.build();
}
}
My IDE complains about this -
The method reader(ItemReader<? extends Scan>) in the type SimpleStepBuilder<Scan,CScan> is not applicable for the arguments (ScanItemReader)
Update #2:
public class CassandraItemProcessor implements ItemProcessor<Scan, CScan> {
@Override
public CScan process(Scan s) throws Exception {
..
return new CScan();
}
}
public class CassandraBatchItemWriter implements ItemWriter<CScan> {
@Override
public void write(List<? extends CScan> arg0) throws Exception {
// TODO Auto-generated method stub
}
}
You can declare your Reader like this
@Component
@JobScope
public class ScanItemReader extends RepositoryItemReader<Scan> {
private final ScanJpaRepository repository;
@Autowired
public ScanItemReader(final ScanJpaRepository repository) {
super();
this.repository = repository;
}
@PostConstruct
protected void init() {
final Map<String, Sort.Direction> sorts = new HashMap<>();
sorts.put("Your sort parameter"), Direction.ASC);// This could be any field name of your Entity class
this.setRepository(this.repository);
this.setSort(sorts);
this.setMethodName(""); // You should sepcify the method which
//spring batch should call in your repository to fetch
// data and the arguments it needs needs to be
//specified with the below method.
// And this method must return Page<T>
this.setArguments();
}
}
Autowire this reader bean and use it in your StepBuilder.
I am trying to do the same but get stucked. If it has worked for you, please share the git repository or any other reference. I want to see how we can read from one db and write into other. I already have JPA methods for it. Just want to autowire and use them.
The technical post webpages of this site follow the CC BY-SA 4.0 protocol. If you need to reprint, please indicate the site URL or the original address.Any question please contact:yoyou2525@163.com.