[英]Spring Boot + Spring Batch + Spring JPA
I am working on Spring Batch job that moves data from Sql Server to Cassandra. 我正在从事将数据从Sql Server移到Cassandra的Spring Batch作业。 I am using Spring Data JPA to read and write the data. 我正在使用Spring Data JPA读取和写入数据。 I have created entities and JPA Repositories for both databases. 我已经为两个数据库创建了实体和JPA存储库。
Now I can not figure out how to use my JpaRepostorty with Spring Batch ItemReader. 现在,我无法弄清楚如何将JpaRepostorty与Spring Batch ItemReader一起使用。 I have searched on internet and found few refs where they mentioned to use JpaPageItemReader. 我在互联网上进行搜索,发现其中提到使用JpaPageItemReader的参考文献很少。 But that requires specifying query and configuring other details. 但这需要指定查询并配置其他详细信息。 But I cannot figure out how to use existing JpaRepository that I have. 但是我无法弄清楚如何使用现有的JpaRepository。 Below is snippet of relevant code- 以下是相关代码的片段-
My JpaRepostory for SQL Server - 我的SQL Server JpaRepostory-
public interface ScanJpaRepository extends JpaRepository<Scan, Integer>
{
@Transactional(readOnly = true)
@Query("select s from Scan s left join fetch s.projectVersion")
Stream<Scan> findAllScan();
}
My Spring Batch Job - 我的春季批处理作业-
@Configuration
@EnableBatchProcessing
public class SSCBatchConfigurationCassandra {
@Autowired
public JobBuilderFactory jobBuilderFactory;
@Autowired
public StepBuilderFactory stepBuilderFactory;
@Bean
public PlatformTransactionManager transactionManager() {
return new ResourcelessTransactionManager();
}
@Bean
public JobExplorer jobExplorer() throws Exception {
MapJobExplorerFactoryBean jobExplorerFactory = new MapJobExplorerFactoryBean(mapJobRepositoryFactoryBean());
jobExplorerFactory.afterPropertiesSet();
return jobExplorerFactory.getObject();
}
@Bean
public MapJobRepositoryFactoryBean mapJobRepositoryFactoryBean() {
MapJobRepositoryFactoryBean mapJobRepositoryFactoryBean = new MapJobRepositoryFactoryBean();
mapJobRepositoryFactoryBean.setTransactionManager(transactionManager());
return mapJobRepositoryFactoryBean;
}
@Bean
public JobRepository jobRepository() throws Exception {
return mapJobRepositoryFactoryBean().getObject();
}
@Bean
public JobLauncher jobLauncher() throws Exception {
SimpleJobLauncher simpleJobLauncher = new SimpleJobLauncher();
simpleJobLauncher.setJobRepository(jobRepository());
return simpleJobLauncher;
}
@Bean
public ItemReader<Project> reader() {
**// how to read from ScanJpaRepository ??**
}
@Bean
public CassandraItemProcessor processor() {
return new CassandraItemProcessor();
}
@Bean
public ItemWriter<CassandraProject> cqlWriter() {
final CassandraBatchItemWriter writer = new CassandraBatchItemWriter();
return writer;
}
// tag::jobstep[]
@Bean
public Job importUserJob(JobCompletionNotificationListener listener, Step step1) {
return jobBuilderFactory.get("importUserJob")
.incrementer(new RunIdIncrementer())
.listener(listener)
.flow(step1)
.end()
.build();
}
@Bean
public Step step1() {
return stepBuilderFactory.get("step1")
.<Project, CassandraProject> chunk(100)
.reader(reader())
.processor(processor())
.writer()
.build();
}
// end::jobstep[]
}
Update #1: I added item reader to my batch job as mentioned. 更新#1:如上所述,我将项目阅读器添加到了批处理作业中。
@Configuration
@EnableBatchProcessing
public class FortifySSCBatchConfigurationCassandra {
....
@Autowired
public ScanItemReader itemReader;
.....
@Bean
public Step step1() {
return stepBuilderFactory.get("step1")
.<Scan, CScan> chunk(100)
.reader(itemReader)
.processor(processor())
.writer(cqlWriter())
.build();
}
}
My IDE complains about this - 我的IDE对此抱怨-
The method reader(ItemReader<? extends Scan>) in the type SimpleStepBuilder<Scan,CScan> is not applicable for the arguments (ScanItemReader)
Update #2: 更新#2:
public class CassandraItemProcessor implements ItemProcessor<Scan, CScan> {
@Override
public CScan process(Scan s) throws Exception {
..
return new CScan();
}
}
public class CassandraBatchItemWriter implements ItemWriter<CScan> {
@Override
public void write(List<? extends CScan> arg0) throws Exception {
// TODO Auto-generated method stub
}
}
You can declare your Reader like this 您可以这样声明您的读者
@Component
@JobScope
public class ScanItemReader extends RepositoryItemReader<Scan> {
private final ScanJpaRepository repository;
@Autowired
public ScanItemReader(final ScanJpaRepository repository) {
super();
this.repository = repository;
}
@PostConstruct
protected void init() {
final Map<String, Sort.Direction> sorts = new HashMap<>();
sorts.put("Your sort parameter"), Direction.ASC);// This could be any field name of your Entity class
this.setRepository(this.repository);
this.setSort(sorts);
this.setMethodName(""); // You should sepcify the method which
//spring batch should call in your repository to fetch
// data and the arguments it needs needs to be
//specified with the below method.
// And this method must return Page<T>
this.setArguments();
}
}
Autowire this reader bean and use it in your StepBuilder. 自动连接此阅读器bean,并在您的StepBuilder中使用它。
I am trying to do the same but get stucked. 我试图做同样的事情,但被卡住了。 If it has worked for you, please share the git repository or any other reference. 如果它对您有用,请共享git存储库或任何其他参考。 I want to see how we can read from one db and write into other. 我想看看我们如何从一个数据库读取并写入另一个数据库。 I already have JPA methods for it. 我已经有JPA方法了。 Just want to autowire and use them. 只想自动接线并使用它们。
声明:本站的技术帖子网页,遵循CC BY-SA 4.0协议,如果您需要转载,请注明本站网址或者原文地址。任何问题请咨询:yoyou2525@163.com.