堆栈内存溢出
  简体   繁体   English

Spring Batch 将资源的值从步骤 1 传递到下一步

[英]Spring batch pass value of resource from step 1 to the next step

 原文  2018-10-18 19:23:58  7   1    java/ spring/ spring-batch

问题描述

So my spring batch job has three steps, the first step(tasklet) will copy files from a shared drive and store them in a folder of the spring project(FileSystems.getDefault()."folder_name").所以我的spring批处理作业有三个步骤，第一步（tasklet）将从共享驱动器复制文件并将它们存储在spring项目的文件夹中（FileSystems.getDefault()."folder_name"）。 The second step should read the xml file from this folder and process as per business logic and finally write into the database.The final step(tasklet) will delete the folder created in step 1. But I am getting " java.lang.IllegalArgumentException: Resources must be set and osbatch.core.step.AbstractStep : Exception while closing step execution resources in step step1 in job job java.lang.NullPointerException: null. " I am using MultiResourceItemReader to read the files in the second step.第二步应该从这个文件夹中读取xml文件并按照业务逻辑进行处理，最后写入数据库。最后一步（tasklet）将删除在步骤1中创建的文件夹。但我得到“ java.lang.IllegalArgumentException：必须设置资源，并且 osbatch.core.step.AbstractStep ：在作业 java.lang.NullPointerException 中的 step1 中关闭步骤执行资源时出现异常：null。 “我在第二步中使用MultiResourceItemReader读取文件。 Can anybody help on how to remove this exception and so that the batch executes as per the mentioned logic.任何人都可以帮助解决如何删除此异常并按照上述逻辑执行批处理。

This is the batch configuration-这是批量配置-

@Autowired
public TaskletStep taskletStep;

@Autowired
public MyItemReader myItemReader;

@Autowired
public MyItemWriter myItemWriter;

@Autowired
public JobBuilderFactory jobBuilderFactory;

@Autowired
public StepBuilderFactory stepBuilderFactory;

@Bean
public Job job() throws FileNotFoundException, XMLStreamException, URISyntaxException {
    return jobBuilderFactory.get("job")
            .incrementer(new RunIdIncrementer())
            .start(step1()).next(step2())
            .build();
}

@Bean
public Step step1() {
    return stepBuilderFactory.get("step1")
            .tasklet(taskletStep).listener(promotionListener())
            .build();
}

    @Bean
    public ExecutionContextPromotionListener promotionListener() {
            ExecutionContextPromotionListener listener = new ExecutionContextPromotionListener();

            listener.setKeys(new String[] {"someKey" });

            return listener;
    };


@Bean
public Step step2() {
    return stepBuilderFactory.get("step2")
            .<String, String> chunk(1)
            .reader(myItemReader)
            .writer(myItemWriter)
            .build();
}

This is the taskletStep这是taskletStep

private static FileSystem fileSystem = FileSystems.getDefault();
private String locationSourceNetwork = "E:\\assss\\wfwef.txt";
private ChunkContext chunkContext;
String homeOfficeDirectoryUnzip =  fileSystem.getPath("HOME").toAbsolutePath().toString();
@Override
public RepeatStatus execute(StepContribution contribution, ChunkContext chunkContext) throws Exception {

    //String homeOfficeDirectoryUnzip =  fileSystem.getPath("HOME").toAbsolutePath().toString();
    File file = new File(homeOfficeDirectoryUnzip);

    Path sourceDirectory = Paths.get(locationSourceNetwork);
    //Path targetDirectory = Paths.get(homeOfficeDirectoryUnzip);
    Files.copy(sourceDirectory, file.toPath());
    ExecutionContext stepContext = chunkContext.getStepContext().getStepExecution().getExecutionContext();
    stepContext.put("someKey", homeOfficeDirectoryUnzip);
    return RepeatStatus.FINISHED;
}
@BeforeStep
public void saveStepExecution(ChunkContext chunkContext) {
    this.chunkContext = chunkContext;
    /*ExecutionContext stepContext = this.stepExecution.getExecutionContext();
    stepContext.put("someKey", homeOfficeDirectoryUnzip);*/
}

This is the MyItemReader-这是 MyItemReader-

private Resource[] resources;私有资源[] 资源；

/*@Autowired
ResourcesFactoryBean fileResources;*/

private static String files =  new ClassPathResource("HOME").toString();//FileSystems.getDefault().getPath("HOME").toAbsolutePath().toString();

private Resource[] resources;


@BeforeStep
public void retrieveInterstepData(StepExecution stepExecution) throws IOException {
    JobExecution jobExecution = stepExecution.getJobExecution();
    ExecutionContext jobContext = jobExecution.getExecutionContext();
    ResourcePatternResolver patternResolver = new PathMatchingResourcePatternResolver();
    this.resources = patternResolver.getResources("file:"+jobContext.get("someKey").toString());
}


public MultiResourceItemReader<String> multiResourceItemReader()
        throws FileNotFoundException, XMLStreamException, URISyntaxException {
    MyItemReader resourceItemReader = new MyItemReader();
    resourceItemReader.setResources(resources);
    resourceItemReader.setDelegate(headerXmlReader());
    resourceItemReader.setStrict(false);
    return resourceItemReader;
}

public StaxEventItemReader<String> headerXmlReader()
        throws FileNotFoundException, XMLStreamException, URISyntaxException {
    // TODO Auto-generated method stub

    StaxEventItemReader<String> staxHeaderDataReader = new StaxEventItemReader<String>();
    System.out.println("HIIIIIIIIIII");
    int i = resources.length;
    while (i > 0) {
        staxHeaderDataReader.setResource(resources[i - 1]);
        i--;
    }
    return staxHeaderDataReader;

}

This resource should have the file which is being copied by TaskletStep but the batch is failing during application startup.此资源应包含 TaskletStep 正在复制但批处理在应用程序启动期间失败的文件。 Note-this is just a dummy poc of the actual scenario.注意——这只是实际场景的一个虚拟 poc。

Stack Trace--
main] o.s.b.c.l.support.SimpleJobLauncher      : Job: [SimpleJob: [name=job]] launched with the following parameters: [{run.id=10}]
2018-10-19 20:35:43.959  INFO 3108 --- [           main] o.s.batch.core.job.SimpleStepHandler     : Executing step: [step1]
2018-10-19 20:35:43.998  INFO 3108 --- [           main] o.s.batch.core.job.SimpleStepHandler     : Executing step: [step2]
2018-10-19 20:35:44.009 ERROR 3108 --- [           main] o.s.batch.core.step.AbstractStep         : Encountered an error executing step step2 in job job

java.lang.IllegalArgumentException: Resources must be set
    at org.springframework.util.Assert.notNull(Assert.java:193) ~[spring-core-5.0.8.RELEASE.jar:5.0.8.RELEASE]
    at org.springframework.batch.item.file.MultiResourceItemReader.open(MultiResourceItemReader.java:168) ~[spring-batch-infrastructure-4.0.1.RELEASE.jar:4.0.1.RELEASE]
    at org.springframework.batch.item.support.CompositeItemStream.open(CompositeItemStream.java:103) ~[spring-batch-infrastructure-4.0.1.RELEASE.jar:4.0.1.RELEASE]
    at org.springframework.batch.core.step.tasklet.TaskletStep.open(TaskletStep.java:310) ~[spring-batch-core-4.0.1.RELEASE.jar:4.0.1.RELEASE]
    at org.springframework.batch.core.step.AbstractStep.execute(AbstractStep.java:197) ~[spring-batch-core-4.0.1.RELEASE.jar:4.0.1.RELEASE]
    at org.springframework.batch.core.job.SimpleStepHandler.handleStep(SimpleStepHandler.java:148) [spring-batch-core-4.0.1.RELEASE.jar:4.0.1.RELEASE]
    at org.springframework.batch.core.job.AbstractJob.handleStep(AbstractJob.java:394) [spring-batch-core-4.0.1.RELEASE.jar:4.0.1.RELEASE]
    at org.springframework.batch.core.job.SimpleJob.doExecute(SimpleJob.java:135) [spring-batch-core-4.0.1.RELEASE.jar:4.0.1.RELEASE]
    at org.springframework.batch.core.job.AbstractJob.execute(AbstractJob.java:308) [spring-batch-core-4.0.1.RELEASE.jar:4.0.1.RELEASE]
    at org.springframework.batch.core.launch.support.SimpleJobLauncher$1.run(SimpleJobLauncher.java:141) [spring-batch-core-4.0.1.RELEASE.jar:4.0.1.RELEASE]
    at org.springframework.core.task.SyncTaskExecutor.execute(SyncTaskExecutor.java:50) [spring-core-5.0.8.RELEASE.jar:5.0.8.RELEASE]
    at org.springframework.batch.core.launch.support.SimpleJobLauncher.run(SimpleJobLauncher.java:134) [spring-batch-core-4.0.1.RELEASE.jar:4.0.1.RELEASE]
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) ~[na:1.8.0_181]
    at sun.reflect.NativeMethodAccessorImpl.invoke(Unknown Source) ~[na:1.8.0_181]
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source) ~[na:1.8.0_181]
    at java.lang.reflect.Method.invoke(Unknown Source) ~[na:1.8.0_181]
    at org.springframework.aop.support.AopUtils.invokeJoinpointUsingReflection(AopUtils.java:343) [spring-aop-5.0.8.RELEASE.jar:5.0.8.RELEASE]
    at org.springframework.aop.framework.ReflectiveMethodInvocation.invokeJoinpoint(ReflectiveMethodInvocation.java:197) [spring-aop-5.0.8.RELEASE.jar:5.0.8.RELEASE]
    at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:163) [spring-aop-5.0.8.RELEASE.jar:5.0.8.RELEASE]
    at org.springframework.batch.core.configuration.annotation.SimpleBatchConfiguration$PassthruAdvice.invoke(SimpleBatchConfiguration.java:127) [spring-batch-core-4.0.1.RELEASE.jar:4.0.1.RELEASE]
    at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:185) [spring-aop-5.0.8.RELEASE.jar:5.0.8.RELEASE]
    at org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:212) [spring-aop-5.0.8.RELEASE.jar:5.0.8.RELEASE]
    at com.sun.proxy.$Proxy89.run(Unknown Source) [na:na]
    at org.springframework.boot.autoconfigure.batch.JobLauncherCommandLineRunner.execute(JobLauncherCommandLineRunner.java:163) [spring-boot-autoconfigure-2.0.4.RELEASE.jar:2.0.4.RELEASE]
    at org.springframework.boot.autoconfigure.batch.JobLauncherCommandLineRunner.executeLocalJobs(JobLauncherCommandLineRunner.java:179) [spring-boot-autoconfigure-2.0.4.RELEASE.jar:2.0.4.RELEASE]
    at org.springframework.boot.autoconfigure.batch.JobLauncherCommandLineRunner.launchJobFromProperties(JobLauncherCommandLineRunner.java:134) [spring-boot-autoconfigure-2.0.4.RELEASE.jar:2.0.4.RELEASE]
    at org.springframework.boot.autoconfigure.batch.JobLauncherCommandLineRunner.run(JobLauncherCommandLineRunner.java:128) [spring-boot-autoconfigure-2.0.4.RELEASE.jar:2.0.4.RELEASE]
    at org.springframework.boot.SpringApplication.callRunner(SpringApplication.java:800) [spring-boot-2.0.4.RELEASE.jar:2.0.4.RELEASE]
    at org.springframework.boot.SpringApplication.callRunners(SpringApplication.java:784) [spring-boot-2.0.4.RELEASE.jar:2.0.4.RELEASE]
    at org.springframework.boot.SpringApplication.run(SpringApplication.java:338) [spring-boot-2.0.4.RELEASE.jar:2.0.4.RELEASE]
    at org.springframework.boot.SpringApplication.run(SpringApplication.java:1258) [spring-boot-2.0.4.RELEASE.jar:2.0.4.RELEASE]
    at org.springframework.boot.SpringApplication.run(SpringApplication.java:1246) [spring-boot-2.0.4.RELEASE.jar:2.0.4.RELEASE]
    at com.infotech.batch.SpringBootBatchTaskletApplication.main(SpringBootBatchTaskletApplication.java:12) [classes/:na]

2018-10-19 20:35:44.014 ERROR 3108 --- [           main] o.s.batch.core.step.AbstractStep         : Exception while closing step execution resources in step step2 in job job

java.lang.NullPointerException: null
    at org.springframework.batch.item.file.MultiResourceItemReader.close(MultiResourceItemReader.java:155) ~[spring-batch-infrastructure-4.0.1.RELEASE.jar:4.0.1.RELEASE]
    at org.springframework.batch.item.support.CompositeItemStream.close(CompositeItemStream.java:89) ~[spring-batch-infrastructure-4.0.1.RELEASE.jar:4.0.1.RELEASE]
    at org.springframework.batch.core.step.tasklet.TaskletStep.close(TaskletStep.java:305) ~[spring-batch-core-4.0.1.RELEASE.jar:4.0.1.RELEASE]
    at org.springframework.batch.core.step.AbstractStep.execute(AbstractStep.java:271) ~[spring-batch-core-4.0.1.RELEASE.jar:4.0.1.RELEASE]
    at org.springframework.batch.core.job.SimpleStepHandler.handleStep(SimpleStepHandler.java:148) [spring-batch-core-4.0.1.RELEASE.jar:4.0.1.RELEASE]
    at org.springframework.batch.core.job.AbstractJob.handleStep(AbstractJob.java:394) [spring-batch-core-4.0.1.RELEASE.jar:4.0.1.RELEASE]
    at org.springframework.batch.core.job.SimpleJob.doExecute(SimpleJob.java:135) [spring-batch-core-4.0.1.RELEASE.jar:4.0.1.RELEASE]
    at org.springframework.batch.core.job.AbstractJob.execute(AbstractJob.java:308) [spring-batch-core-4.0.1.RELEASE.jar:4.0.1.RELEASE]
    at org.springframework.batch.core.launch.support.SimpleJobLauncher$1.run(SimpleJobLauncher.java:141) [spring-batch-core-4.0.1.RELEASE.jar:4.0.1.RELEASE]
    at org.springframework.core.task.SyncTaskExecutor.execute(SyncTaskExecutor.java:50) [spring-core-5.0.8.RELEASE.jar:5.0.8.RELEASE]
    at org.springframework.batch.core.launch.support.SimpleJobLauncher.run(SimpleJobLauncher.java:134) [spring-batch-core-4.0.1.RELEASE.jar:4.0.1.RELEASE]
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) ~[na:1.8.0_181]
    at sun.reflect.NativeMethodAccessorImpl.invoke(Unknown Source) ~[na:1.8.0_181]
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source) ~[na:1.8.0_181]
    at java.lang.reflect.Method.invoke(Unknown Source) ~[na:1.8.0_181]
    at org.springframework.aop.support.AopUtils.invokeJoinpointUsingReflection(AopUtils.java:343) [spring-aop-5.0.8.RELEASE.jar:5.0.8.RELEASE]
    at org.springframework.aop.framework.ReflectiveMethodInvocation.invokeJoinpoint(ReflectiveMethodInvocation.java:197) [spring-aop-5.0.8.RELEASE.jar:5.0.8.RELEASE]
    at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:163) [spring-aop-5.0.8.RELEASE.jar:5.0.8.RELEASE]
    at org.springframework.batch.core.configuration.annotation.SimpleBatchConfiguration$PassthruAdvice.invoke(SimpleBatchConfiguration.java:127) [spring-batch-core-4.0.1.RELEASE.jar:4.0.1.RELEASE]
    at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:185) [spring-aop-5.0.8.RELEASE.jar:5.0.8.RELEASE]
    at org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:212) [spring-aop-5.0.8.RELEASE.jar:5.0.8.RELEASE]
    at com.sun.proxy.$Proxy89.run(Unknown Source) [na:na]
    at org.springframework.boot.autoconfigure.batch.JobLauncherCommandLineRunner.execute(JobLauncherCommandLineRunner.java:163) [spring-boot-autoconfigure-2.0.4.RELEASE.jar:2.0.4.RELEASE]
    at org.springframework.boot.autoconfigure.batch.JobLauncherCommandLineRunner.executeLocalJobs(JobLauncherCommandLineRunner.java:179) [spring-boot-autoconfigure-2.0.4.RELEASE.jar:2.0.4.RELEASE]
    at org.springframework.boot.autoconfigure.batch.JobLauncherCommandLineRunner.launchJobFromProperties(JobLauncherCommandLineRunner.java:134) [spring-boot-autoconfigure-2.0.4.RELEASE.jar:2.0.4.RELEASE]
    at org.springframework.boot.autoconfigure.batch.JobLauncherCommandLineRunner.run(JobLauncherCommandLineRunner.java:128) [spring-boot-autoconfigure-2.0.4.RELEASE.jar:2.0.4.RELEASE]
    at org.springframework.boot.SpringApplication.callRunner(SpringApplication.java:800) [spring-boot-2.0.4.RELEASE.jar:2.0.4.RELEASE]
    at org.springframework.boot.SpringApplication.callRunners(SpringApplication.java:784) [spring-boot-2.0.4.RELEASE.jar:2.0.4.RELEASE]
    at org.springframework.boot.SpringApplication.run(SpringApplication.java:338) [spring-boot-2.0.4.RELEASE.jar:2.0.4.RELEASE]
    at org.springframework.boot.SpringApplication.run(SpringApplication.java:1258) [spring-boot-2.0.4.RELEASE.jar:2.0.4.RELEASE]
    at org.springframework.boot.SpringApplication.run(SpringApplication.java:1246) [spring-boot-2.0.4.RELEASE.jar:2.0.4.RELEASE]
    at com.infotech.batch.SpringBootBatchTaskletApplication.main(SpringBootBatchTaskletApplication.java:12) [classes/:na]

2018-10-19 20:35:44.019  INFO 3108 --- [           main] o.s.b.c.l.support.SimpleJobLauncher      : Job: [SimpleJob: [name=job]] completed with the following parameters: [{run.id=10}] and the following status: [FAILED]

1 个解决方案

解决方案1
 0  2018-10-19 08:32:55

You don't need to set the resource manually on the delegate reader here:您无需在此处的委托阅读器上手动设置资源：

private StaxEventItemReader<String> headerXmlReader()
    throws FileNotFoundException, XMLStreamException, URISyntaxException {
   // TODO Auto-generated method stub

   StaxEventItemReader<String> staxHeaderDataReader = new StaxEventItemReader<String>();

   int i = resources.length;
   while (i > 0) {
       staxHeaderDataReader.setResource(resources[i - 1]);
       i--;
   }
   return staxHeaderDataReader;

}

The MultiResourceItemReader will do it automatically while reading the resources. MultiResourceItemReader将在读取资源时自动执行此操作。 Basically, what you need to do is to configure everything on the delegate (root tag name, unmarshaller, etc) except the resource.基本上，您需要做的是在委托上配置除资源之外的所有内容（根标记名称、解组器等）。 The resource will be set dynamically by the MultiResourceItemReader on the delegate.资源将由委托上的MultiResourceItemReader动态设置。

You can find an example of how to configure a MultiResourceItemReader with a delegate StaxEventItemReader here .您可以在此处找到有关如何使用委托StaxEventItemReader配置MultiResourceItemReader的StaxEventItemReader 。

pass value of resource from step 1 to the next step将资源值从步骤 1 传递到下一步

step1 should dynamically pass the information about the created resource to step2 via the step execution context. step1 应该通过 step 执行上下文将有关创建的资源的信息动态传递给 step2。 For more details about how to do this, see:有关如何执行此操作的更多详细信息，请参阅：

Hope this helps.希望这可以帮助。

暂无
暂无

声明:本站的技术帖子网页，遵循CC BY-SA 4.0协议，如果您需要转载，请注明本站网址或者原文地址。任何问题请咨询:yoyou2525@163.com.

相关问题 Spring Batch：将读入文件的名称传递给下一步 - Spring Batch: Pass over names of read-in files to the next step 一步之内的Spring Batch - Spring Batch within a Step Spring Batch分区步骤 - Spring Batch partitioning a step Spring 批处理步骤未执行 - Spring Batch step not executing Spring批处理中的验证步骤 - Validation step in Spring batch 如何从 Spring 批处理作业中终止步骤 - How to terminate step from a Spring batch job Spring 批处理如何在 ExecutionContext 中设置文件路径以进行下一步 - Spring Batch how to set filepath in ExecutionContext for next step 春季批处理：填充n个文件，但下一步显示为空 - Spring Batch: Fill n files but the next step sees empty 如何获得Spring Batch作业的下一步中上一步执行的ID？ - How do I get the ID of the previous step execution in the next step of a spring batch job? 将Spring Batch Tasklet失败消息传递给报告步骤。 - Pass Spring Batch Tasklet Failure Message to Reporting Step.
相关标签
 
粤ICP备18138465号  © 2020-2024 STACKOOM.COM