Spring Batch not executing StepExecutionListener beforeStep or afterStep methods

Question

Update: I was re-running my code and actually noticed none of my Listeners are actually working...

I have a Spring Batch application and I am overriding the StepExecutionListener and providing my own implementation. I am registering it with the TaskletStep, however, I never see the log messages that the beforeStep/afterStep methods should be outputting:

MyStepExecutionListener.java

public class MyStepExecutionListener implements StepExecutionListener {

 @Override
    public void beforeStep(StepExecution stepExecution) {
        // begin my own custom implementation
        LOGGER.info("Before the step!");
    }

    @Override
    public ExitStatus afterStep(StepExecution stepExecution) {
         // begin my own custom implementation
         LOGGER.info("After the step!");
        return stepExecution.getStatus();

    }
}

I have defined my Tasklet Step in my BatchConfig.java class as:

public BatchConfig {

@Bean
    public static org.springframework.batch.core.scope.JobScope jobScope() {
        org.springframework.batch.core.scope.JobScope jobScope = new org.springframework.batch.core.scope.JobScope();
        jobScope.setProxyTargetClass(true);
        jobScope.setAutoProxy(true);
        return jobScope;
    }

    @Bean
    public static org.springframework.batch.core.scope.StepScope stepScope() {
        org.springframework.batch.core.scope.StepScope stepScope = new org.springframework.batch.core.scope.StepScope();
        stepScope.setProxyTargetClass(true);
        stepScope.setAutoProxy(true);
        return stepScope;
    }

    @Bean
//    @StepScope
    public StepExecutionListener stepExecutionListener() {
        return new MyStepExecutionListener();
    }

 @Bean
    @Qualifier("s3FlatfFileReaderForMktgOffrs")
    @StepScope
    public S3FlatFileItemReader<FieldSet> s3FlatfFileReaderForMktgOffrs() {
        return new S3FlatFileItemReader<>(lineMapper());
    }

 @Bean
    @Qualifier("s3FlatfFileReaderCustom")
    @StepScope
    public S3FlatFileItemReader<FieldSet> s3FlatfFileReaderCustom() {
        // Custom class that Extends FlatFileItemReader
        return new S3FlatFileItemReader<>(lineMapper());
    }

  @Bean
    @Qualifier("myCustomFileItemReader")
    @StepScope
    public ItemStreamReader<List<FieldSet>> myCustomFileItemReader(
            @Value("#{jobParameters}") Map jobParameters) {

        String fileName = (String) jobParameters.get("fileName");
        String region = (String) jobParameters.get("region");
        String bucketName = awsS3EastBucket;
        if (StringUtils.equals(region, Regions.US_WEST_2.getName())) {
            bucketName = awsS3WestBucket;
        }

        // Custom class that Extends FlatFileItemReader
        S3FlatFileItemReader<FieldSet> s3FileItemReader = s3FlatfFileReaderCustom();

     
         s3FileItemReader.setResource(S3_PROTOCOL_PREFIX + bucketName + SLASH + fileName);
        }

        s3FileItemReader.setStrict(false);
        s3FileItemReader.setLinesToSkip(1);
        s3FileItemReader.setSaveState(false);

        AggregateItemReader aggregateItemReader = new AggregateItemReader(s3FileItemReader) {
            @Override
            protected String getItemKey(FieldSet item) {
                return item.readString(FIRST_NAME) + "-" +
                        item.readString(LAST_NAME);
            }
        };
        SynchronizedItemStreamReader<List<FieldSet>> fieldSetSynchronizedItemStreamReader = new SynchronizedItemStreamReader<>();
        fieldSetSynchronizedItemStreamReader.setDelegate(aggregateItemReader);
        return fieldSetSynchronizedItemStreamReader;
    }



@Bean(name = "myCustomStep")
    @Scope("prototype")
    @SuppressWarnings("unchecked")
    public Step myCustomStep(PlatformTransactionManager transactionManager) {
        TaskletStep step = stepBuilderFactory.get("myCustomStep")
                .<List<FieldSet>, List<MyPayLoadRecord>>chunk(250)
                .reader(myCustomFileItemReader(OVERRIDDEN_BY_EXPRESSION))
                .processor(myCustomProcessor())
                .writer(myCustomWriter())
                .faultTolerant()
                .skipPolicy(new AlwaysSkipItemSkipPolicy())
                .skip(DataValidationException.class)
                .listener(stepExecutionListener())
                .listener(new CustomReaderListener())
                .listener(new CustomProcessListener())
                .listener(new CustomWriteListener())
                .listener(new CustomSkipListener())
                .taskExecutor(batchTaskExecutor())
                .throttleLimit(maxThreads)
                .build();
        step.setTransactionManager(transactionManager);
        //step.registerStepExecutionListener(stepExecutionListener());
        step.registerChunkListener(new CustomChunkListener());
        return step;
    }
}

I have commented out step.registerStepExecutionListener(stepExecutionListener()); and tried setting the listener as seen above, but neither implementation worked. I was under the impression I should just implement the StepExecutionListener then register it with the TaskletStep - am I missing something here?

Answer 1

I do not understand the reason for making your step as a prototype-scoped bean, but I'm not able to reproduce the issue with a setup similar to what you shared (using Spring Batch v4.3.3). Here is a quick example:

import java.util.Arrays;

import org.springframework.batch.core.ExitStatus;
import org.springframework.batch.core.Job;
import org.springframework.batch.core.JobParameters;
import org.springframework.batch.core.Step;
import org.springframework.batch.core.StepExecution;
import org.springframework.batch.core.StepExecutionListener;
import org.springframework.batch.core.configuration.annotation.EnableBatchProcessing;
import org.springframework.batch.core.configuration.annotation.JobBuilderFactory;
import org.springframework.batch.core.configuration.annotation.StepBuilderFactory;
import org.springframework.batch.core.configuration.annotation.StepScope;
import org.springframework.batch.core.launch.JobLauncher;
import org.springframework.batch.item.support.ListItemReader;
import org.springframework.context.ApplicationContext;
import org.springframework.context.annotation.AnnotationConfigApplicationContext;
import org.springframework.context.annotation.Bean;
import org.springframework.context.annotation.Configuration;
import org.springframework.context.annotation.Scope;

@Configuration
@EnableBatchProcessing
public class SO68217675 {

    @Bean
//    @StepScope // works with a step-scoped bean as well
    public StepExecutionListener stepExecutionListener() {
        return new MyStepExecutionListener();
    }

    @Bean
//    @Scope("prototype") // works with a prototype bean as well, but I don't understand the reason for that scope
    public Step step(StepBuilderFactory stepBuilderFactory) {
        return stepBuilderFactory.get("step")
                .<Integer, Integer>chunk(2)
                .reader(new ListItemReader<>(Arrays.asList(1, 2, 3, 4)))
                .writer(items -> items.forEach(System.out::println))
                .listener(stepExecutionListener())
                .build();
    }

    @Bean
    public Job job(JobBuilderFactory jobBuilderFactory, StepBuilderFactory stepBuilderFactory) {
        return jobBuilderFactory.get("job")
                .start(step(stepBuilderFactory))
                .build();
    }

    static class MyStepExecutionListener implements StepExecutionListener {

        @Override
        public void beforeStep(StepExecution stepExecution) {
            System.out.println("MyStepExecutionListener.beforeStep");
        }

        @Override
        public ExitStatus afterStep(StepExecution stepExecution) {
            System.out.println("MyStepExecutionListener.afterStep");
            return ExitStatus.COMPLETED;
        }
    }

    public static void main(String[] args) throws Exception {
        ApplicationContext context = new AnnotationConfigApplicationContext(SO68217675.class);
        JobLauncher jobLauncher = context.getBean(JobLauncher.class);
        Job job = context.getBean(Job.class);
        jobLauncher.run(job, new JobParameters());
    }

}

This example prints the following (as expected):

MyStepExecutionListener.beforeStep
1
2
3
4
MyStepExecutionListener.afterStep

Answer 2

Per @MahmoudBenHassine answer - By implementing the Listener (Step or any of them) as a static nested class, I was able to use the Slf4j LOGGER already in my BatchConfiguration.java class, and see that indeed the Listeners were working . Though, I was still perplexed why this seem to not work when I defined the Listener in its own package/class. I then started to suspect the LOGGERS, as basically, this was how I was verifying whether it was "working or not". Thus, I placed some System.out.println("") dummy output in my StepExecutionerListener (the one I was having issues with that was defined in its own custom Class/package) and lo-and-behold , I saw the output.

Thus, the Listeners were working the entire time , however, the LOGGERS were NOT working, and my validation methodology was inferior, as it was shadowing/hiding the main problem (using the LOGGERS as proof of whether the Step/Chunk/Job Listener was working or not).

Per this post, it made me more curious something was wrong with my slf4j/log4j dependencies in my POM or maybe I was missing something:

Spring Batch produces no logging output when launched from the command line

Furthermore, I setup my POM.xml in a way for logging per this Article, which recommended removing all the spring-boot-starter-logging from ALL the spring-boot-starters (which I did), and adding 1 spring-boot-starter-log4j2 dependency if you plan to use log4j (like I did):

https://www.baeldung.com/spring-boot-logging

Anyway, I removed the <exclusion> for the spring-boot-starter-logging inside my spring-boot-starter-batch dependency defined in my POM and the Logging is working fine now (it seems) in the custom class Listeners (Step/Chunk/Job) now.

When I examine my project dependencies in IntelliJ I have:

log4j:log4j:1.2.15
org.apache.logging.log4j:log4j-api:2.13.3
org.apache.logging.log4j:log4j-core:2.13.3
org.apache.logging.log4j:log4j-jul:2.13.3
org.apache.logging.log4j:log4j-slf4j-impl:2.13.3
org.slf4j:jcl-over-slf4j:1.7.30
org.slf4j:jul-to-slf4j:1.7.30
org.slf4j:log4j-over-slf4j:1.7.30
org.slf4j:slf4j-api:1.7.30

I noticed by removing the exclusion on the spring-boot-starter-batch, as mentioned above, I saw this dependency get added:

org.apache.logging.log4j:log4j-to-slf4j:2.13.3

So I'm assuming Spring Batch need's this when triggering the listeners before/after methods for LOGGING. Hope this helps anyone!