[英]Spring Batch - My Batch seems executing two steps at the same time?
I can't really understand what's going on.我真的不明白发生了什么。 I'm studying Spring Batch and I'd like to execute two steps for some reasons, one after the other.
我正在研究 Spring Batch,由于某些原因,我想一个接一个地执行两个步骤。
Now please don't mind what the steps are currently doing, just keep in mind that I would like to perform two steps sequentially.现在请不要介意这些步骤当前在做什么,只要记住我想按顺序执行两个步骤。
This is the code:这是代码:
@Configuration
@EnableBatchProcessing
public class JobConfiguration {
@Autowired
private JobBuilderFactory jobBuilderFactory;
@Autowired
private StepBuilderFactory stepBuilderFactory;
private List<Employee> employeesToSave = new ArrayList<Employee>();
public JsonItemReader<Employee> jsonReader() {
System.out.println("Try to read JSON");
final ObjectMapper mapper = new ObjectMapper();
final JacksonJsonObjectReader<Employee> jsonObjectReader = new JacksonJsonObjectReader<>(
Employee.class);
jsonObjectReader.setMapper(mapper);
return new JsonItemReaderBuilder<Employee>().jsonObjectReader(jsonObjectReader)
.resource(new ClassPathResource("input.json"))
.name("myReader")
.build();
}
public ListItemReader<Employee> listReader() {
System.out.println("Read from list");
return new ListItemReader<Employee>(employeesToSave);
*/
}
public ItemProcessor<Employee,Employee> filterProcessor() {
return employee -> {
System.out.println("Processing JSON");
return employee;
};
}
public ItemWriter<Employee> filterWriter() {
return listEmployee -> {
employeesToSave.addAll(listEmployee);
System.out.println("Save on list " + listEmployee.toString());
};
}
public ItemWriter<Employee> insertToDBWriter() {
System.out.println("Try to save on DB");
return listEmployee -> {
System.out.println("Save on DB " + listEmployee.toString());
};
}
public Step filterStep() {
StepBuilder stepBuilder = stepBuilderFactory.get("filterStep");
SimpleStepBuilder<Employee, Employee> simpleStepBuilder = stepBuilder.chunk(5);
return simpleStepBuilder.reader(jsonReader()).processor(filterProcessor()).writer(filterWriter()).build();
}
public Step insertToDBStep() {
StepBuilder stepBuilder = stepBuilderFactory.get("insertToDBStep");
SimpleStepBuilder<Employee, Employee> simpleStepBuilder = stepBuilder.chunk(5);
return simpleStepBuilder.reader(listReader()).writer(insertToDBWriter()).build();
}
@Bean
public Job myJob(JobRepository jobRepository, PlatformTransactionManager platformTransactionManager) {
return jobBuilderFactory.get("myJob").incrementer(new RunIdIncrementer())
.start(filterStep())
.next(insertToDBStep())
.build();
}
}
Why doesn't the insertToDBStep starts at the end of the filterStep and it actually looks like the filter is running at the same time?为什么 insertToDBStep 不在 filterStep 的末尾开始,实际上看起来过滤器正在同时运行? And why it looks like the job starts after the initialization of Root WebApplicationContext?
为什么它看起来像在初始化 Root WebApplicationContext 之后开始工作?
This is the output.这是输出。
2022-05-23 15:40:49.418 INFO 14008 --- [ restartedMain] w.s.c.ServletWebServerApplicationContext : Root WebApplicationContext: initialization completed in 1024 ms
Try to read JSON
Read from list
Try to save on DB
2022-05-23 15:40:49.882 INFO 14008 --- [ restartedMain] o.s.b.d.a.OptionalLiveReloadServer : LiveReload server is running on port 35729
2022-05-23 15:40:49.917 INFO 14008 --- [ restartedMain] o.s.b.w.embedded.tomcat.TomcatWebServer : Tomcat started on port(s): 8080 (http) with context path ''
2022-05-23 15:40:49.926 INFO 14008 --- [ restartedMain] c.marco.firstbatch.TestBatchApplication : Started TestBatchApplication in 1.985 seconds (JVM running for 2.789)
2022-05-23 15:40:49.927 INFO 14008 --- [ restartedMain] o.s.b.a.b.JobLauncherApplicationRunner : Running default command line with: []
2022-05-23 15:40:49.928 WARN 14008 --- [ restartedMain] o.s.b.c.c.a.DefaultBatchConfigurer : No datasource was provided...using a Map based JobRepository
2022-05-23 15:40:49.928 WARN 14008 --- [ restartedMain] o.s.b.c.c.a.DefaultBatchConfigurer : No transaction manager was provided, using a ResourcelessTransactionManager
2022-05-23 15:40:49.943 INFO 14008 --- [ restartedMain] o.s.b.c.l.support.SimpleJobLauncher : No TaskExecutor has been set, defaulting to synchronous executor.
2022-05-23 15:40:49.972 INFO 14008 --- [ restartedMain] o.s.b.c.l.support.SimpleJobLauncher : Job: [SimpleJob: [name=myJob]] launched with the following parameters: [{run.id=1}]
2022-05-23 15:40:50.003 INFO 14008 --- [ restartedMain] o.s.batch.core.job.SimpleStepHandler : Executing step: [filterStep]
Processing JSON
Processing JSON
Processing JSON
Processing JSON
Processing JSON
Save on list [com.marco.firstbatch.Employee@958d6e7, com.marco.firstbatch.Employee@464d17f8, com.marco.firstbatch.Employee@705520ac, com.marco.firstbatch.Employee@1a9f8e93, com.marco.firstbatch.Employee@55bf8cc9]
Processing JSON
Processing JSON
Save on list [com.marco.firstbatch.Employee@55d706c0, com.marco.firstbatch.Employee@1bc46dd4]
2022-05-23 15:40:50.074 INFO 14008 --- [ restartedMain] o.s.batch.core.step.AbstractStep : Step: [filterStep] executed in 70ms
2022-05-23 15:40:50.081 INFO 14008 --- [ restartedMain] o.s.batch.core.job.SimpleStepHandler : Executing step: [insertToDBStep]
2022-05-23 15:40:50.084 INFO 14008 --- [ restartedMain] o.s.batch.core.step.AbstractStep : Step: [insertToDBStep] executed in 3ms
2022-05-23 15:40:50.088 INFO 14008 --- [ restartedMain] o.s.b.c.l.support.SimpleJobLauncher : Job: [SimpleJob: [name=myJob]] completed with the following parameters: [{run.id=1}] and the following status: [COMPLETED] in 96ms
Thanks in advance.提前致谢。
The steps are executed correctly in sequence.这些步骤按顺序正确执行。 You are putting
System.out.println
statements in two "kind" of places:您将
System.out.println
语句放在两个“种类”的地方:
In your case, Spring Framework will call the following bean definition methods in order to define the first step, filterStep()
:在您的情况下,Spring Framework 将调用以下 bean 定义方法来定义第一步
filterStep()
:
jsonReader()
: prints Try to read JSON
. jsonReader()
:打印Try to read JSON
。 The file is not read at this time, only the json reader bean is defined.json reader bean created
.json reader bean created
。listReader()
: prints Read from list
. listReader()
:打印Read from list
。 Same here, the file reading has not started yet.list reader bean created
.list reader bean created
。filterProcessor()
: prints nothing. filterProcessor()
:什么都不打印。 The log statement is in the ItemProcessor#process
method.ItemProcessor#process
方法中。 This will be called by Spring Batch at runtime, not at this point in time which is a configuration timefilterWriter()
: same here, the print statement is in the write method called at runtime and not at configuration time filterWriter()
:这里一样,打印语句在运行时调用的 write 方法中,而不是在配置时调用This results in the following output for filterStep()
:这导致
filterStep()
的以下输出:
Try to read JSON
Read from list
Now Spring Framework moves to defining the next step, insertToDBStep()
.现在 Spring Framework 开始定义下一步,
insertToDBStep()
。 For this, it will call the following methods in order, according to your step definition:为此,它将根据您的步骤定义按顺序调用以下方法:
listReader()
: this bean has already bean defined, Spring will reuse the same instance (by default, Spring beans are singletons). listReader()
:这个 bean 已经定义了 bean,Spring 将重用相同的实例(默认情况下,Spring bean 是单例)。 Hence, there is no output from this method.insertToDBWriter()
: prints Try to save on DB
. insertToDBWriter()
:打印Try to save on DB
。 Same here, there is no actual save to DB here.insertToDBWriter bean created
(or even more accurate, attempting to create insertToDBWriter bean
, in case the code that follows throws an exception).insertToDBWriter bean created
(或者更准确地说, attempting to create insertToDBWriter bean
,以防后面的代码抛出异常)。 You now have the following cumulative output:您现在有以下累积输出:
Try to read JSON
Read from list
Try to save on DB
At this point, Spring Framework has finished its job of configuring the application context, Spring Batch takes over and starts the job.至此,Spring Framework 已经完成了配置应用上下文的工作,Spring Batch 接管并启动了这项工作。 The actual processing of
filterStep()
begins: filterStep()
的实际处理开始:
ListItemReader
) does not have any output in the read
method.ListItemReader
) 在read
方法中没有任何输出。Processing JSON
Processing JSON
Save on list ...
Save on list ...
You seem to have two chunks (the first with 5 items and the second with 2 items), which leads to the following output:您似乎有两个块(第一个有 5 个项目,第二个有 2 个项目),这导致以下输出:
2022-05-23 15:40:50.003 INFO 14008 --- [ restartedMain] o.s.batch.core.job.SimpleStepHandler : Executing step: [filterStep]
Processing JSON
Processing JSON
Processing JSON
Processing JSON
Processing JSON
Save on list [com.marco.firstbatch.Employee@958d6e7, com.marco.firstbatch.Employee@464d17f8, com.marco.firstbatch.Employee@705520ac, com.marco.firstbatch.Employee@1a9f8e93, com.marco.firstbatch.Employee@55bf8cc9]
Processing JSON
Processing JSON
Save on list [com.marco.firstbatch.Employee@55d706c0, com.marco.firstbatch.Employee@1bc46dd4]
2022-05-23 15:40:50.074 INFO 14008 --- [ restartedMain] o.s.batch.core.step.AbstractStep : Step: [filterStep] executed in 70ms
Then, the next step starts its execution and you get the following output:然后,下一步开始执行,您将获得以下输出:
2022-05-23 15:40:50.081 INFO 14008 --- [ restartedMain] o.s.batch.core.job.SimpleStepHandler : Executing step: [insertToDBStep]
2022-05-23 15:40:50.084 INFO 14008 --- [ restartedMain] o.s.batch.core.step.AbstractStep : Step: [insertToDBStep] executed in 3ms
Here you might be asking why there are no items written by insertToDBWriter()
(ie why there are no Save on DB ..
logs).在这里,您可能会问为什么
insertToDBWriter()
没有写入任何项目(即为什么没有Save on DB ..
日志)。 This is because the listReader()
is a singleton bean and you are using it in both steps, so when the second step calls its read
method, it will still return null
, because the same instance is used and which has already exhausted the list of items in step 1. Hence, this step ends immediately since there are no items to process.这是因为
listReader()
是一个单例 bean,并且您在两个步骤中都在使用它,所以当第二步调用它的read
方法时,它仍然会返回null
,因为使用了相同的实例并且已经用尽了步骤 1 中的项目。因此,此步骤立即结束,因为没有要处理的项目。 If you want to re-read the items from the list in the second step, you can annotate the reader method with @StepScope
.如果要在第二步中重新读取列表中的项目,可以使用
@StepScope
注释 reader 方法。 This will create a distinct instance of the reader for each step.这将为每个步骤创建一个不同的阅读器实例。
声明:本站的技术帖子网页,遵循CC BY-SA 4.0协议,如果您需要转载,请注明本站网址或者原文地址。任何问题请咨询:yoyou2525@163.com.