How to resolve NullPointerException in MultiResourceItemReader Spring Batch

Question

I'm reading multiple files from the S3 bucket using MultiResourceItemReader , I'm getting NullPointerException before executing the myReader() method, however, after throwing an exception it's going into the myReader() method but itemProcessor method is not calling. I believe due to NullPointerException itemProcessor not invoking. Something wrong with MultiResourceItemReader not sure what's going wrong here.

Please find my code below:

        @Bean
        public MultiResourceItemReader<String> multiResourceReader()
        {
            String bucket = "mybucket;
            String key = "/myfiles";
        
            List<InputStream> resourceList = s3Client.getFiles(bucket, key);
            List<InputStreamResource> inputStreamResourceList = new ArrayList<>();
            for (InputStream s: resourceList) {
                inputStreamResourceList.add(new InputStreamResource(s));
            }
    
    Resource[] resources = inputStreamResourceList.toArray(new InputStreamResource[inputStreamResourceList.size()]);
//InputStreamResource[] resources = inputStreamResourceList.toArray(new InputStreamResource[inputStreamResourceList.size()]);
    
    // I'm getting all the stream content - I verified my stream is not null
            for (int i = 0; i < resources.length; i++) {
                try {
                    InputStream s  = resources[i].getInputStream();
                    String result = IOUtils.toString(s, StandardCharsets.UTF_8);
                    System.out.println(result);
                } catch (IOException e) {
                    e.printStackTrace();
                }
            }
    
            MultiResourceItemReader<String> resourceItemReader = new MultiResourceItemReader<>();
            resourceItemReader.setResources(resources);
            resourceItemReader.setDelegate(myReader());
            return resourceItemReader;
        }


    @Bean
    @StepScope
    public FlatFileItemReader<String> myReader() {
    
        FlatFileItemReader reader = new FlatFileItemReader<>();
        reader.setLineMapper(new DefaultLineMapper());
        return reader;
    }

Exception:

java.lang.NullPointerException: null
        at org.springframework.batch.item.file.MultiResourceItemReader$1.compare(MultiResourceItemReader.java:83)
        at org.springframework.batch.item.file.MultiResourceItemReader$1.compare(MultiResourceItemReader.java:76)
        at java.base/java.util.TimSort.countRunAndMakeAscending(TimSort.java:355)
        at java.base/java.util.TimSort.sort(TimSort.java:220)
        at java.base/java.util.Arrays.sort(Arrays.java:1441)
        at org.springframework.batch.item.file.MultiResourceItemReader.open(MultiResourceItemReader.java:186)
        at org.springframework.batch.item.support.CompositeItemStream.open(CompositeItemStream.java:104)
        at org.springframework.batch.core.step.tasklet.TaskletStep.open(TaskletStep.java:311)
        at org.springframework.batch.core.step.AbstractStep.execute(AbstractStep.java:205)
        at org.springframework.batch.core.job.SimpleStepHandler.handleStep(SimpleStepHandler.java:152)
        at org.springframework.batch.core.job.flow.JobFlowExecutor.executeStep(JobFlowExecutor.java:68)
        at org.springframework.batch.core.job.flow.support.state.StepState.handle(StepState.java:68)
        at org.springframework.batch.core.job.flow.support.SimpleFlow.resume(SimpleFlow.java:169)
        at org.springframework.batch.core.job.flow.support.SimpleFlow.start(SimpleFlow.java:144)
        at org.springframework.batch.core.job.flow.FlowJob.doExecute(FlowJob.java:137)
        at org.springframework.batch.core.job.AbstractJob.execute(AbstractJob.java:320)
        at org.springframework.batch.core.launch.support.SimpleJobLauncher$1.run(SimpleJobLauncher.java:149)
        at org.springframework.core.task.SyncTaskExecutor.execute(SyncTaskExecutor.java:50)
        at org.springframework.batch.core.launch.support.SimpleJobLauncher.run(SimpleJobLauncher.java:140)
        at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
        at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.base/java.lang.reflect.Method.invoke(Method.java:566)
        at org.springframework.aop.support.AopUtils.invokeJoinpointUsingReflection(AopUtils.java:344)
        at org.springframework.aop.framework.ReflectiveMethodInvocation.invokeJoinpoint(ReflectiveMethodInvocation.java:198)
        at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:163)
        at org.springframework.batch.core.configuration.annotation.SimpleBatchConfiguration$PassthruAdvice.invoke(SimpleBatchConfiguration.java:128)
        at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:186)
        at org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:215)
        at com.sun.proxy.$Proxy80.run(Unknown Source)
        at org.springframework.boot.autoconfigure.batch.JobLauncherApplicationRunner.execute(JobLauncherApplicationRunner.java:199)
        at org.springframework.boot.autoconfigure.batch.JobLauncherApplicationRunner.executeLocalJobs(JobLauncherApplicationRunner.java:173)
        at org.springframework.boot.autoconfigure.batch.JobLauncherApplicationRunner.launchJobFromProperties(JobLauncherApplicationRunner.java:160)
        at org.springframework.boot.autoconfigure.batch.JobLauncherApplicationRunner.run(JobLauncherApplicationRunner.java:155)
        at org.springframework.boot.autoconfigure.batch.JobLauncherApplicationRunner.run(JobLauncherApplicationRunner.java:150)
        at org.springframework.boot.SpringApplication.callRunner(SpringApplication.java:782)
        at org.springframework.boot.SpringApplication.callRunners(SpringApplication.java:772)
        at org.springframework.boot.SpringApplication.run(SpringApplication.java:345)
        at org.springframework.boot.SpringApplication.run(SpringApplication.java:1343)
        at org.springframework.boot.SpringApplication.run(SpringApplication.java:1332)
        at 
        at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
        at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.base/java.lang.reflect.Method.invoke(Method.java:566)
        at org.springframework.boot.loader.MainMethodRunner.run(MainMethodRunner.java:49)
        at org.springframework.boot.loader.Launcher.launch(Launcher.java:108)
        at org.springframework.boot.loader.Launcher.launch(Launcher.java:58)
        at org.springframework.boot.loader.JarLauncher.main(JarLauncher.java:88)

Can someone please help me to resolve this issue. Appreciated your help in advance. Thanks.

Answer 1

The reason you see the NullPointerException is due to the default comparator used by the MultiResourceItemReader to sort the resources after loading them.

The default compare behavior calls the getFilename() method of the InputStreamResource.

Refer - https://github.com/spring-projects/spring-batch/blob/115c3022147692155d45e23cdd5cef84895bf9f5/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/file/MultiResourceItemReader.java#L82

But the InputStreamResource just inherits the getFileName() method from its parent AbstractResource , which just returns null. https://github.com/spring-projects/spring-framework/blob/316e84f04f3dbec3ea5ab8563cc920fb21f49749/spring-core/src/main/java/org/springframework/core/io/AbstractResource.java#L220

The solution is to provide a custom comparator for the MultiResourceItemReader. Here is a simple example, assuming you do not want to sort the resources in a specific way before processing:

public class CustomComparator implements Comparator<InputStream>{

        @Override
        public int compare(InputStream is1, InputStream is2) {
       //comparing based on last modified time
            return Long.compare(is1.hashCode(),is2.hashCode());
   }
}

MultiResourceItemReader<String> resourceItemReader = new MultiResourceItemReader<>();
resourceItemReader.setResources(resources);
resourceItemReader.setDelegate(myReader());
resourceItemReader.setDelegate(new CustomComparator());

Refer this answer for how a Comparator is used by Spring Batch MultiResourceItemReader.

File processing order with Spring Batch