How to pass parameters from ItemReader to ItemProcessor in spring-batch?

Question

Question is simple: how can I pass a value that is only know within ItemReader (eg the current filename) to the ItemProcessor ?

@Bean
@JobScope
public ItemReader<String> reader(@Value("#{jobParameters['path]}") String path) {
    FlatFileItemReader<String> delegate = new FlatFileItemReader<>();
    delegate.setLineMapper(new PassThroughLineMapper());

    Resource[] res = new PathMatchingResourcePatternResolver().getResources("file:" + path);

    MultiResourceItemReader<String> r = new MultiResourceItemReader<>();
    r.setResources(res);
    r.setDelegate(delegate);
    return r;
}

@Bean
public ItemProcessor<String, String> processor() {
    return new ItemProcessor<String, String>() {

        @Override
        public String process(String item) throws Exception {
            //TODO I need the filename that is currently processed. HOW?
            return null;
        }
    };
}

Answer 1

in order to be on the safe side, you should add the name of the file to the object that is returned by the reader. In order to do that, you have to implement your own wrapper Reader. Something like this:

public class MyReader {
     private MultiResourceItemReader delegatereader;

     public MyContainerDto read() {
          String line = delegatereader.read();
          if (line==null) return null;

          Resource currentResource = delegatereader.getCurrentResource();

          MyContainerDto container = MyContainerDto();
          container.setLine(line);
          container.setResourceName(currentResource.getFileName());
          return container;
     }

     ...
} 

(this code was not tested, it just illustrates the aproach I would take)

On my laptop, I'm able to crate 1 million objects within a second, so the additional performance needed in order to create the objects isn't really something that affects the overall performance significantly.

The problem is, that the reader reads as many items, as is defined by the chunksize. After that, it will call the processor for every item in this chunk. Therefore, within a chunk, the items in it could have been read from different files. Hence, there is no other way than to bind the line and the filename together in the reader.

see also https://blog.codecentric.de/en/2012/03/transactions-in-spring-batch-part-1-the-basics/

Answer 2

It's probably best to introduce a dto for the reader.

public class ResourceAwareString implements ResourceAware {
     private String content;
     private Resource res;

     @Override
     public void setResource(Resource res) {
         this.res = res;
     }
}

Then the MultiResourceItemReader<ResourceAwareString> will automatically detect it and write the resource to the dto.

Other approach would be to make both ItemReader and ItemProcessor @StepScope and use @BeforeStep to get access to the current StepExection context. Then it's possible to set/get a value, and thereby transport values from reader to processer, or even to writer.

Note that @BeforeStep only works on a public class, and NOT on an anonymous method like:

new FlatFileItemReader<String>() {
  @BeforeStep
  public void before(StepExecution stepEx) {
       //this will never be called! don't do this.
  }
};