想要在 Spring Batch 中为事务实现指数退避

Question

Basically the title. I am using a JDBC item reader and JDBC item writer and I'm changing a particular status through the processor by using an API, if the API fails to change the status, I want to use exponential backoff to retry this at a later instant. I'm not able to figure out how to implement this

Answer 1

You have two options:

1. Handle the retry operation manually in your item processor

There are basically two ways to do that, either programmatically or in a declarative way.

1.1 Programmatic approach

You first define your retry template with a backoff policy as needed:

@Bean
public RetryTemplate retryTemplate() {
    // configure backoff policy
    ExponentialBackOffPolicy exponentialBackOffPolicy = new ExponentialBackOffPolicy();
    exponentialBackOffPolicy.setInitialInterval(1000);
    exponentialBackOffPolicy.setMultiplier(2.0);
    exponentialBackOffPolicy.setMaxInterval(10000);

    // configure retry policy
    SimpleRetryPolicy simpleRetryPolicy = new SimpleRetryPolicy();
    simpleRetryPolicy.setMaxAttempts(5);

    // configure retry template
    RetryTemplate retryTemplate = new RetryTemplate();
    retryTemplate.setBackOffPolicy(exponentialBackOffPolicy);
    retryTemplate.setRetryPolicy(simpleRetryPolicy);

    return retryTemplate;
}

Then use that retry template in your item processor:

import org.springframework.batch.item.ItemProcessor;
import org.springframework.retry.RetryCallback;
import org.springframework.retry.RetryContext;
import org.springframework.retry.support.RetryTemplate;

public class MyRetryableItemProcessor implements ItemProcessor {

    RetryTemplate retryTemplate;

    public MyRetryableItemProcessor(RetryTemplate retryTemplate) {
        this.retryTemplate = retryTemplate;
    }

    @Override
    public Object process(Object item) throws Exception {
        return retryTemplate.execute(new RetryCallback<Object, Exception>() {
            @Override
            public Object doWithRetry(RetryContext retryContext) throws Exception {
                // API call
                return item;
            }
        });
    }
}

1.2 Declarative approach using annotations

Here is an example:

import org.springframework.batch.item.ItemProcessor;
import org.springframework.retry.RetryCallback;
import org.springframework.retry.RetryContext;
import org.springframework.retry.annotation.Backoff;
import org.springframework.retry.annotation.Retryable;
import org.springframework.retry.support.RetryTemplate;
import org.springframework.stereotype.Component;

@Component
public class MyAnnotationBasedRetryableItemProcessor implements ItemProcessor {

    @Override
    @Retryable(backoff = @Backoff(delay = 1000L, maxDelay = 10000, multiplier = 2.0))
    public Object process(Object item) throws Exception {
        // Do API call
        return item;
    }

}

2. Let Spring Batch handle the retry for you by using a fault-tolerant step

In this case, you can set a custom RetryPolicy in your fault-tolerant step:

@Bean
public Step step(StepBuilderFactory stepBuilderFactory) {
    // configure backoff policy
    ExponentialBackOffPolicy exponentialBackOffPolicy = new ExponentialBackOffPolicy();
    exponentialBackOffPolicy.setInitialInterval(1000);
    exponentialBackOffPolicy.setMultiplier(2.0);
    exponentialBackOffPolicy.setMaxInterval(10000);

    // configure retry policy
    SimpleRetryPolicy simpleRetryPolicy = new SimpleRetryPolicy();
    simpleRetryPolicy.setMaxAttempts(5);
    
    return stepBuilderFactory.get("myStep")
            .<Integer, Integer>chunk(5)
            .reader(itemReader())
            .processor(itemProcessor())
            .writer(itemWriter())
            .faultTolerant()
            .retryPolicy(simpleRetryPolicy)
            .build();
}

Note that in this case, whenever your processor throws an exception for an item, the entire chunk is retried item by item (and each item will be re-processed in its own transaction).

---

The examples above use spring-retry since you mentioned you have a preference for that. But the same ideas can be applied with any other fault-tolerance library.