[英]Spring-Batch without persisting metadata to database?
我想創建一個spring-batch
作業,但我想在沒有任何數據庫持久性的情況下運行它。 不幸的是,spring-batch 需要以某種方式將作業周期的metadata
寫入數據庫,從而促使我至少提供某種帶有事務管理器和實體管理器的數據庫。
是否可以阻止元數據並獨立於 txmanagers 和數據庫運行?
更新:
ERROR org.springframework.batch.core.job.AbstractJob: Encountered fatal error executing job
java.lang.NullPointerException
at org.springframework.batch.core.repository.dao.MapJobExecutionDao.synchronizeStatus(MapJobExecutionDao.java:158) ~[spring-batch-core-3.0.1.RELEASE.jar:3.0.1.RELEASE]
at org.springframework.batch.core.repository.support.SimpleJobRepository.update(SimpleJobRepository.java:161) ~[spring-batch-core-3.0.1.RELEASE.jar:3.0.1.RELEASE]
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) ~[?:1.7.0_51]
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) ~[?:1.7.0_51]
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[?:1.7.0_51]
at java.lang.reflect.Method.invoke(Method.java:606) ~[?:1.7.0_51]
at org.springframework.aop.support.AopUtils.invokeJoinpointUsingReflection(AopUtils.java:317) ~[spring-aop-4.0.6.RELEASE.jar:4.0.6.RELEASE]
at org.springframework.aop.framework.ReflectiveMethodInvocation.invokeJoinpoint(ReflectiveMethodInvocation.java:190) ~[spring-aop-4.0.6.RELEASE.jar:4.0.6.RELEASE]
at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:157) ~[spring-aop-4.0.6.RELEASE.jar:4.0.6.RELEASE]
at org.springframework.transaction.interceptor.TransactionInterceptor$1.proceedWithInvocation(TransactionInterceptor.java:98) ~[spring-tx-4.0.6.RELEASE.jar:4.0.6.RELEASE]
at org.springframework.transaction.interceptor.TransactionAspectSupport.invokeWithinTransaction(TransactionAspectSupport.java:262) ~[spring-tx-4.0.6.RELEASE.jar:4.0.6.RELEASE]
at org.springframework.transaction.interceptor.TransactionInterceptor.invoke(TransactionInterceptor.java:95) ~[spring-tx-4.0.6.RELEASE.jar:4.0.6.RELEASE]
at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:179) ~[spring-aop-4.0.6.RELEASE.jar:4.0.6.RELEASE]
at org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:207) ~[spring-aop-4.0.6.RELEASE.jar:4.0.6.RELEASE]
at com.sun.proxy.$Proxy134.update(Unknown Source) ~[?:?]
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) ~[?:1.7.0_51]
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) ~[?:1.7.0_51]
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[?:1.7.0_51]
at java.lang.reflect.Method.invoke(Method.java:606) ~[?:1.7.0_51]
at org.springframework.aop.support.AopUtils.invokeJoinpointUsingReflection(AopUtils.java:317) ~[spring-aop-4.0.6.RELEASE.jar:4.0.6.RELEASE]
at org.springframework.aop.framework.ReflectiveMethodInvocation.invokeJoinpoint(ReflectiveMethodInvocation.java:190) ~[spring-aop-4.0.6.RELEASE.jar:4.0.6.RELEASE]
at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:157) ~[spring-aop-4.0.6.RELEASE.jar:4.0.6.RELEASE]
at org.springframework.batch.core.configuration.annotation.SimpleBatchConfiguration$PassthruAdvice.invoke(SimpleBatchConfiguration.java:127) ~[spring-batch-core-3.0.1.RELEASE.jar:3.0.1.RELEASE]
at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:179) ~[spring-aop-4.0.6.RELEASE.jar:4.0.6.RELEASE]
at org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:207) ~[spring-aop-4.0.6.RELEASE.jar:4.0.6.RELEASE]
at com.sun.proxy.$Proxy134.update(Unknown Source) ~[?:?]
at org.springframework.batch.core.job.AbstractJob.updateStatus(AbstractJob.java:416) ~[spring-batch-core-3.0.1.RELEASE.jar:3.0.1.RELEASE]
at org.springframework.batch.core.job.AbstractJob.execute(AbstractJob.java:299) [spring-batch-core-3.0.1.RELEASE.jar:3.0.1.RELEASE]
at org.springframework.batch.core.launch.support.SimpleJobLauncher$1.run(SimpleJobLauncher.java:135) [spring-batch-core-3.0.1.RELEASE.jar:3.0.1.RELEASE]
at org.springframework.core.task.SyncTaskExecutor.execute(SyncTaskExecutor.java:50) [spring-core-4.0.6.RELEASE.jar:4.0.6.RELEASE]
at org.springframework.batch.core.launch.support.SimpleJobLauncher.run(SimpleJobLauncher.java:128) [spring-batch-core-3.0.1.RELEASE.jar:3.0.1.RELEASE]
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) ~[?:1.7.0_51]
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) ~[?:1.7.0_51]
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[?:1.7.0_51]
at java.lang.reflect.Method.invoke(Method.java:606) ~[?:1.7.0_51]
at org.springframework.aop.support.AopUtils.invokeJoinpointUsingReflection(AopUtils.java:317) [spring-aop-4.0.6.RELEASE.jar:4.0.6.RELEASE]
at org.springframework.aop.framework.ReflectiveMethodInvocation.invokeJoinpoint(ReflectiveMethodInvocation.java:190) [spring-aop-4.0.6.RELEASE.jar:4.0.6.RELEASE]
at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:157) [spring-aop-4.0.6.RELEASE.jar:4.0.6.RELEASE]
at org.springframework.batch.core.configuration.annotation.SimpleBatchConfiguration$PassthruAdvice.invoke(SimpleBatchConfiguration.java:127) [spring-batch-core-3.0.1.RELEASE.jar:3.0.1.RELEASE]
at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:179) [spring-aop-4.0.6.RELEASE.jar:4.0.6.RELEASE]
at org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:207) [spring-aop-4.0.6.RELEASE.jar:4.0.6.RELEASE]
at com.sun.proxy.$Proxy50.run(Unknown Source) [?:?]
只需為批處理配置創建一個沒有數據源的配置:
@Configuration
@EnableAutoConfiguration
@EnableBatchProcessing
public class BatchConfiguration extends DefaultBatchConfigurer {
@Override
public void setDataSource(DataSource dataSource) {
// override to do not set datasource even if a datasource exist.
// initialize will use a Map based JobRepository (instead of database)
}
}
它將使用基於內存映射的實現來初始化 JobRepository 和 JobExplorer。 https://github.com/spring-projects/spring-batch/blob/342d27bc1ed83312bdcd9c0cb30510f4c469e47d/spring-batch-core/src/main/java/org/springframework/batch/core/configuration/annotation/DefaultBatchConfigurer.java#L84
即使使用 spring 啟動自動配置,您也可以使用您的生產數據源。
我想在沒有任何數據庫持久性的情況下運行它
您可以使用MapJobRepositoryFactoryBean
和ResourcelessTransactionManager
示例配置:
<bean id="transactionManager"
class="org.springframework.batch.support.transaction.ResourcelessTransactionManager" />
<bean id="jobRepository"
class="org.springframework.batch.core.repository.support.MapJobRepositoryFactoryBean">
<property name="transactionManager" ref="transactionManager" />
</bean>
<bean id="jobLauncher"
class="org.springframework.batch.core.launch.support.SimpleJobLauncher">
<property name="jobRepository" ref="jobRepository" />
</bean>
對於 Spring 4.X,基於注解的配置如下:
@Bean
public PlatformTransactionManager getTransactionManager() {
return new ResourcelessTransactionManager();
}
@Bean
public JobRepository getJobRepo() {
return new MapJobRepositoryFactoryBean(getTransactionManager()).getObject();
}
調整@Braj 的答案后,我的工作配置如下所示:
@Bean
public ResourcelessTransactionManager transactionManager() {
return new ResourcelessTransactionManager();
}
@Bean
public JobRepository jobRepository(ResourcelessTransactionManager transactionManager) throws Exception {
MapJobRepositoryFactoryBean mapJobRepositoryFactoryBean = new MapJobRepositoryFactoryBean(transactionManager);
mapJobRepositoryFactoryBean.setTransactionManager(transactionManager);
return mapJobRepositoryFactoryBean.getObject();
}
@Bean
public SimpleJobLauncher jobLauncher(JobRepository jobRepository) {
SimpleJobLauncher simpleJobLauncher = new SimpleJobLauncher();
simpleJobLauncher.setJobRepository(jobRepository);
return simpleJobLauncher;
}
我回到我自己的問題,因為該解決方案不再有效。 從 spring-batch-1.5.3 開始,使用如下:
@SpringBootApplication(exclude = {DataSourceAutoConfiguration.class})
...
@Bean
public PlatformTransactionManager transactionManager() {
return new ResourcelessTransactionManager();
}
}
如果不想在數據庫中存儲作業的元數據,配置如下
@Configuration
public class InMemoryJobRepositoryConfiguration {
@Bean
public ResourcelessTransactionManager transactionManager() {
return new ResourcelessTransactionManager();
}
@Bean
public MapJobRepositoryFactoryBean mapJobRepositoryFactory(ResourcelessTransactionManager transactionManager)
throws Exception {
MapJobRepositoryFactoryBean factory = new MapJobRepositoryFactoryBean(transactionManager);
factory.afterPropertiesSet();
return factory;
}
@Bean
public JobRepository jobRepository(MapJobRepositoryFactoryBean repositoryFactory) throws Exception {
return repositoryFactory.getObject();
}
@Bean
public JobExplorer jobExplorer(MapJobRepositoryFactoryBean repositoryFactory) {
return new SimpleJobExplorer(repositoryFactory.getJobInstanceDao(), repositoryFactory.getJobExecutionDao(),
repositoryFactory.getStepExecutionDao(), repositoryFactory.getExecutionContextDao());
}
@Bean
public SimpleJobLauncher jobLauncher(JobRepository jobRepository) {
SimpleJobLauncher launcher = new SimpleJobLauncher();
launcher.setJobRepository(jobRepository);
return launcher;
}
}
如果job需要讀寫業務數據到數據庫,數據源配置如下。
@Autowired
private DataSource dataSource;
spring batch 使用該數據源創建內部模式(BATCH_* 表和序列)。 為了防止這種情況發生,在 application.properties 中設置標志spring.batch.initializer.enabled=false
。
我嘗試了上述所有解決方案,但它不適用於我的特定場景,因為我的 spring 批處理作業具有多線程等高級功能。 它需要一個數據庫。 所以這是我為解決問題所做的工作:
@Configuration
public class FakeBatchConfig implements BatchConfigurer {
PlatformTransactionManager transactionManager;
JobRepository jobRepository;
JobLauncher jobLauncher;
JobExplorer jobExplorer;
@Override
public JobRepository getJobRepository() {
return jobRepository;
}
@Override
public PlatformTransactionManager getTransactionManager() {
return transactionManager;
}
@Override
public JobLauncher getJobLauncher() {
return jobLauncher;
}
@Override
public JobExplorer getJobExplorer() {
return jobExplorer;
}
@PostConstruct
void initialize() throws Exception {
if (this.transactionManager == null) {
this.transactionManager = new ResourcelessTransactionManager();
}
MapJobRepositoryFactoryBean jobRepositoryFactory = new MapJobRepositoryFactoryBean(this.transactionManager);
jobRepositoryFactory.afterPropertiesSet();
this.jobRepository = jobRepositoryFactory.getObject();
MapJobExplorerFactoryBean jobExplorerFactory = new MapJobExplorerFactoryBean(jobRepositoryFactory);
jobExplorerFactory.afterPropertiesSet();
this.jobExplorer = jobExplorerFactory.getObject();
this.jobLauncher = createJobLauncher();
}
private JobLauncher createJobLauncher() throws Exception {
SimpleJobLauncher jobLauncher = new SimpleJobLauncher();
jobLauncher.setJobRepository(jobRepository);
jobLauncher.afterPropertiesSet();
return jobLauncher;
}
}
除了@Braj-s 響應之外,我還必須排除批量自動配置: @SpringBootApplication(exclude = {BatchAutoConfiguration.class})
否則它無法正常工作。 這包括 Spring Boot 應用程序
在閱讀了所有提供的答案后,我得到了它的工作。 我混合使用了membersound 和Aure77 解決方案。 我發現不需要覆蓋 setDataSource 方法。 DefaultBatchConfigurer 會自動處理 PlatformTransactionManager 並且不需要 DataSource。 請注意我使用的是 Spring Boot 2.0.3.RELEASE。 以下是我使用的方法
@SpringBootApplication(exclude = { DataSourceAutoConfiguration.class })
@EnableBatchProcessing
public class DemoApplication extends DefaultBatchConfigurer {
@Autowired
private JobBuilderFactory jobs;
@Autowired
private StepBuilderFactory steps;
@Bean
protected Tasklet tasklet() {
return new Tasklet() {
@Override
public RepeatStatus execute(StepContribution contribution, ChunkContext context) {
return RepeatStatus.FINISHED;
}
};
}
@Bean
public Job job() throws Exception {
return this.jobs.get("job").start(step1()).build();
}
@Bean
protected Step step1() throws Exception {
return this.steps.get("step1").tasklet(tasklet()).build();
}
public static void main(String[] args) throws Exception {
//System.exit(SpringApplication.exit(SpringApplication.run(DemoApplication.class, args)));
SpringApplication.run(DemoApplication.class, args);
}
}
最簡單的是將嵌入式數據庫添加到類路徑。 自然不推薦在生產中使用,但如果您使用它來學習,請節省時間。
將嵌入的 H2 數據庫依賴項添加到 pom.xml:
<dependency>
<groupId>com.h2database</groupId>
<artifactId>h2</artifactId>
<scope>runtime</scope>
</dependency>
就這樣。
可以進行以下更改以使用 Spring Batch,而無需創建與 Spring Batch 相關的元數據數據庫結構。
這對我有用:
ResourcelessTransactionManager
類型的 bean 並配置了MapJobRepositoryFactoryBean
。MapJobRepositoryFactoryBean
創建了一個SimpleJoblauncher
bean。@EnableBatchProcessing
下面添加了注釋@SpringBootApplication
。spring.main.allow-bean-definition-overriding=true
在application.properties
。
聲明:本站的技術帖子網頁,遵循CC BY-SA 4.0協議,如果您需要轉載,請注明本站網址或者原文地址。任何問題請咨詢:yoyou2525@163.com.