Add parameter to job context from tasklet step and use in later steps in Spring Batch

Question

For now, I'm using jobParameters to get the filenames for both my FlatFileItemReader and FlatFileItemWriter. It's okay for testing my batch, but my goal is to read a file in some directory (there is only this file in this directory) and the filename might change. The output filename should depend on the input filename.

Therefore, I thought about adding a new step to my job, and this step will set both output and input filenames by searching the good directory and looking for the file into it. I read Passing Data to Future Steps from Spring Doc, and this thread from SO, but I can't make it work, the files are always "null".

First, I've defined the following Tasklet

public class SettingFilenamesTasklet implements Tasklet {

    private StepExecution stepExecution;

    @Override
    public RepeatStatus execute(StepContribution contribution, ChunkContext chunkContext) throws Exception {
        // TODO Search folder and set real filenames
        String inputFilename = "D:/TestInputFolder/dataFile.csv";
        String outputFilename = "D:/TestOutputFolder/dataFile-processed.csv";
        ExecutionContext stepContext = stepExecution.getExecutionContext();
        stepContext.put("inputFile", inputFilename);
        stepContext.put("outputFile", outputFilename);
        return RepeatStatus.FINISHED;
    }

    @BeforeStep
    public void saveStepExecution(StepExecution stepExec) {
        stepExecution = stepExec;
    }
}

Then, I added the promotionListener bean

@Bean
public ExecutionContextPromotionListener promotionListener() {
    ExecutionContextPromotionListener listener = new ExecutionContextPromotionListener();
    listener.setKeys(new String[]{
            "inputFile", "outputFile"
    });
    return listener;
}

I changed the jobParameters by a jobExecutionContext in my FlatFileItemWriter definition (I didn't change a single line to the code itself)

@Bean
@StepScope
public FlatFileItemWriter<RedevableCRE> flatFileWriter(@Value("#{jobExecutionContext[outputFile]}") String outputFile) {
    FlatFileItemWriter<Employee> flatWriter = new FlatFileItemWriter<Employee>();
    FileSystemResource isr;
    isr = new FileSystemResource(new File(outputFile));
    flatWriter.setResource(isr);
    DelimitedLineAggregator<RedevableCRE> aggregator = new DelimitedLineAggregator<RedevableCRE>();
    aggregator.setDelimiter(";");
    BeanWrapperFieldExtractor<RedevableCRE> beanWrapper = new BeanWrapperFieldExtractor<RedevableCRE>();
    beanWrapper.setNames(new String[]{
        "id", "firstName", "lastName", "phone", "address"
    });
    aggregator.setFieldExtractor(beanWrapper);
    flatWriter.setLineAggregator(aggregator);
    flatWriter.setEncoding("ISO-8859-1");
    return flatWriter;
}

I added my Tasklet bean

@Bean
public SettingFilenamesTasklet settingFilenames() {
    return new SettingFilenamesTasklet();
}

And I created a new Step to add in my job declaration

@Bean
public Step stepSettings(StepBuilderFactory stepBuilderFactory, SettingFilenamesTasklet tasklet, ExecutionContextPromotionListener listener) {
    return stepBuilderFactory.get("stepSettings").tasklet(tasklet).listener(listener).build();
}

For now, the FlatFileItemReader still uses the jobParameters value, I want to make my FlatFileItemWriter work first. I get the following error :

[...]    
Caused by: org.springframework.beans.BeanInstantiationException: Failed to instantiate [org.springframework.batch.item.file.FlatFileItemWriter]: Factory method 'flatFileWriter' threw exception; nested exception is java.lang.NullPointerException
    at org.springframework.beans.factory.support.SimpleInstantiationStrategy.instantiate(SimpleInstantiationStrategy.java:189)
    at org.springframework.beans.factory.support.ConstructorResolver.instantiateUsingFactoryMethod(ConstructorResolver.java:591)
    ... 87 common frames omitted
Caused by: java.lang.NullPointerException: null
    at java.io.File.<init>(Unknown Source)
    at batchTest.BatchConfiguration.flatFileWriter(BatchConfiguration.java:165)
    at batchTest.BatchConfiguration$$EnhancerBySpringCGLIB$$5d415889.CGLIB$flatFileWriter$1(<generated>)
    at batchTest.BatchConfiguration$$EnhancerBySpringCGLIB$$5d415889$$FastClassBySpringCGLIB$$969a8527.invoke(<generated>)
    at org.springframework.cglib.proxy.MethodProxy.invokeSuper(MethodProxy.java:228)
    at org.springframework.context.annotation.ConfigurationClassEnhancer$BeanMethodInterceptor.intercept(ConfigurationClassEnhancer.java:312)
    at batchTest.BatchConfiguration$$EnhancerBySpringCGLIB$$5d415889.flatFileWriter(<generated>)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(Unknown Source)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source)
    at java.lang.reflect.Method.invoke(Unknown Source)
    at org.springframework.beans.factory.support.SimpleInstantiationStrategy.instantiate(SimpleInstantiationStrategy.java:162)
    ... 88 common frames omitted

I tried to replace the @StepScope annotation by @JobScope ; to put my parameters directly into jobExecutionContext (+ JobExecutionListener) instead of using StepContext + promotionListener... Nothing works. The resource file is always null when I try to create the FlatFileItemWriter.

What am I missing ?

Thanks for your help.

Answer 1

In tasklet you have ChunkContext at your disposal so you do not need @BeforeStep , you can remove it (in my configuration it is not invoked at all, and when you think of it as one action step does not make much sense but I do not see NPE so guess that part work). We solved it with one of two approaches:

1. You can put any parameter from tasklet to job ExecutionContext directly using chunkContext.getStepContext().getStepExecution().getJobExecution().getExecutionContext().put("inputFile", inputFilename);

2. You can add ExecutionContextPromotionListener to your tasklet step and then do chunkContext.getStepContext().getStepExecution().getExecutionContext().put("inputFile", inputFilename);