使用 CompletableFuture 重試邏輯

Question

我需要在我正在處理的異步框架中提交任務，但我需要捕獲異常，並在“中止”之前多次重試同一任務。

我正在使用的代碼是：

int retries = 0;
public CompletableFuture<Result> executeActionAsync() {

    // Execute the action async and get the future
    CompletableFuture<Result> f = executeMycustomActionHere();

    // If the future completes with exception:
    f.exceptionally(ex -> {
        retries++; // Increment the retry count
        if (retries < MAX_RETRIES)
            return executeActionAsync();  // <--- Submit one more time

        // Abort with a null value
        return null;
    });

    // Return the future    
    return f;
}

這目前無法編譯，因為 lambda 的返回類型是錯誤的：它需要一個Result ，但executeActionAsync返回一個CompletableFuture<Result> 。

我怎樣才能實現這個完全異步的重試邏輯？

Answer 1

鏈接后續重試可以很簡單：

public CompletableFuture<Result> executeActionAsync() {
    CompletableFuture<Result> f=executeMycustomActionHere();
    for(int i=0; i<MAX_RETRIES; i++) {
        f=f.exceptionally(t -> executeMycustomActionHere().join());
    }
    return f;
}

^{閱讀下面的缺點}
這只是按照預期鏈接盡可能多的重試，因為這些后續階段在非異常情況下不會做任何事情。

一個缺點是，如果第一次嘗試立即失敗，那么在鏈接第一個exceptionally處理程序時f已經異常完成，則調用線程將調用該操作，從而完全消除請求的異步性質。 通常， join()可能會阻塞一個線程（默認執行器將啟動一個新的補償線程，但仍然不鼓勵這樣做）。 不幸的是，沒有一個exceptionallyAsync或一個exceptionallyCompose方法。

不調用join()解決方案是

public CompletableFuture<Result> executeActionAsync() {
    CompletableFuture<Result> f=executeMycustomActionHere();
    for(int i=0; i<MAX_RETRIES; i++) {
        f=f.thenApply(CompletableFuture::completedFuture)
           .exceptionally(t -> executeMycustomActionHere())
           .thenCompose(Function.identity());
    }
    return f;
}

演示如何將“組合”和“異常”處理程序結合起來。

此外，如果所有重試都失敗，則只會報告最后一個異常。 更好的解決方案應該報告第一個異常，並將重試的后續異常添加為抑制異常。 這樣的解決方案可以通過鏈接遞歸調用來構建，正如Gili 的回答所暗示的那樣，但是，為了使用這個想法進行異常處理，我們必須使用上面顯示的組合“組合”和“異常”的步驟：

public CompletableFuture<Result> executeActionAsync() {
    return executeMycustomActionHere()
        .thenApply(CompletableFuture::completedFuture)
        .exceptionally(t -> retry(t, 0))
        .thenCompose(Function.identity());
}
private CompletableFuture<Result> retry(Throwable first, int retry) {
    if(retry >= MAX_RETRIES) return CompletableFuture.failedFuture(first);
    return executeMycustomActionHere()
        .thenApply(CompletableFuture::completedFuture)
        .exceptionally(t -> { first.addSuppressed(t); return retry(first, retry+1); })
        .thenCompose(Function.identity());
}

CompletableFuture.failedFuture是一種 Java 9 方法，但如果需要，將 Java 8 兼容的向后移植添加到您的代碼中將是微不足道的：

public static <T> CompletableFuture<T> failedFuture(Throwable t) {
    final CompletableFuture<T> cf = new CompletableFuture<>();
    cf.completeExceptionally(t);
    return cf;
}

Answer 2

我想我成功了。 這是我創建的示例類和測試代碼：

---

可重試任務

public class RetriableTask
{
    protected static final int MAX_RETRIES = 10;
    protected int retries = 0;
    protected int n = 0;
    protected CompletableFuture<Integer> future = new CompletableFuture<Integer>();

    public RetriableTask(int number) {
        n = number;
    }

    public CompletableFuture<Integer> executeAsync() {
        // Create a failure within variable timeout
        Duration timeoutInMilliseconds = Duration.ofMillis(1*(int)Math.pow(2, retries));
        CompletableFuture<Integer> timeoutFuture = Utils.failAfter(timeoutInMilliseconds);

        // Create a dummy future and complete only if (n > 5 && retries > 5) so we can test for both completion and timeouts. 
        // In real application this should be a real future
        final CompletableFuture<Integer> taskFuture = new CompletableFuture<>();
        if (n > 5 && retries > 5)
            taskFuture.complete(retries * n);

        // Attach the failure future to the task future, and perform a check on completion
        taskFuture.applyToEither(timeoutFuture, Function.identity())
            .whenCompleteAsync((result, exception) -> {
                if (exception == null) {
                    future.complete(result);
                } else {
                    retries++;
                    if (retries >= MAX_RETRIES) {
                        future.completeExceptionally(exception);
                    } else {
                        executeAsync();
                    }
                }
            });

        // Return the future    
        return future;
    }
}

---

用法

int size = 10;
System.out.println("generating...");
List<RetriableTask> tasks = new ArrayList<>();
for (int i = 0; i < size; i++) {
    tasks.add(new RetriableTask(i));
}

System.out.println("issuing...");
List<CompletableFuture<Integer>> futures = new ArrayList<>();
for (int i = 0; i < size; i++) {
    futures.add(tasks.get(i).executeAsync());
}

System.out.println("Waiting...");
for (int i = 0; i < size; i++) {
    try {
        CompletableFuture<Integer> future = futures.get(i);
        int result = future.get();
        System.out.println(i + " result is " + result);
    } catch (Exception ex) {
        System.out.println(i + " I got exception!");
    }
}
System.out.println("Done waiting...");

---

輸出

generating...
issuing...
Waiting...
0 I got exception!
1 I got exception!
2 I got exception!
3 I got exception!
4 I got exception!
5 I got exception!
6 result is 36
7 result is 42
8 result is 48
9 result is 54
Done waiting...

---

主要思想和一些膠水代碼（ failAfter函數）來自這里。

歡迎任何其他建議或改進。

Answer 3

我建議不要使用自己的重試邏輯，而是使用經過驗證的庫，如failsafe ，它內置了對期貨的支持（並且似乎比guava-retrying更受歡迎）。 對於您的示例，它看起來像：

private static RetryPolicy retryPolicy = new RetryPolicy()
    .withMaxRetries(MAX_RETRIES);

public CompletableFuture<Result> executeActionAsync() {
    return Failsafe.with(retryPolicy)
        .with(executor)
        .withFallback(null)
        .future(this::executeMycustomActionHere);
}

可能您應該避免.withFallback(null)並讓返回的未來的.get()方法拋出結果異常，以便您的方法的調用者可以專門處理它，但這是您必須做出的設計決定。

其他需要考慮的事情包括是否應該立即重試或在兩次嘗試之間等待一段時間、任何類型的遞歸退避（當您調用可能關閉的 Web 服務時很有用），以及是否存在特定的異常值得重試（例如，如果方法的參數無效）。

Answer 4

工具類：

public class RetryUtil {

    public static <R> CompletableFuture<R> retry(Supplier<CompletableFuture<R>> supplier, int maxRetries) {
        CompletableFuture<R> f = supplier.get();
        for(int i=0; i<maxRetries; i++) {
            f=f.thenApply(CompletableFuture::completedFuture)
                .exceptionally(t -> {
                    System.out.println("retry for: "+t.getMessage());
                    return supplier.get();
                })
                .thenCompose(Function.identity());
        }
        return f;
    }
}

用法：

public CompletableFuture<String> lucky(){
    return CompletableFuture.supplyAsync(()->{
        double luckNum = Math.random();
        double luckEnough = 0.6;
        if(luckNum < luckEnough){
            throw new RuntimeException("not luck enough: " + luckNum);
        }
        return "I'm lucky: "+luckNum;
    });
}
@Test
public void testRetry(){
    CompletableFuture<String> retry = RetryUtil.retry(this::lucky, 10);
    System.out.println("async check");
    String join = retry.join();
    System.out.println("lucky? "+join);
}

輸出

async check
retry for: java.lang.RuntimeException: not luck enough: 0.412296354211683
retry for: java.lang.RuntimeException: not luck enough: 0.4099777199676573
lucky? I'm lucky: 0.8059089479049389

Answer 5

我最近使用guava-retrying庫解決了一個類似的問題。

Callable<Result> callable = new Callable<Result>() {
    public Result call() throws Exception {
        return executeMycustomActionHere();
    }
};

Retryer<Boolean> retryer = RetryerBuilder.<Result>newBuilder()
        .retryIfResult(Predicates.<Result>isNull())
        .retryIfExceptionOfType(IOException.class)
        .retryIfRuntimeException()
        .withStopStrategy(StopStrategies.stopAfterAttempt(MAX_RETRIES))
        .build();

CompletableFuture.supplyAsync( () -> {
    try {
        retryer.call(callable);
    } catch (RetryException e) {
        e.printStackTrace();
    } catch (ExecutionException e) {
       e.printStackTrace();
    }
});

Answer 6

這是一種適用於任何CompletionStage子類的方法，並且不會返回一個虛擬的CompletableFuture ，它只會等待其他期貨更新。

/**
 * Sends a request that may run as many times as necessary.
 *
 * @param request  a supplier initiates an HTTP request
 * @param executor the Executor used to run the request
 * @return the server response
 */
public CompletionStage<Response> asyncRequest(Supplier<CompletionStage<Response>> request, Executor executor)
{
    return retry(request, executor, 0);
}

/**
 * Sends a request that may run as many times as necessary.
 *
 * @param request  a supplier initiates an HTTP request
 * @param executor the Executor used to run the request
 * @param tries    the number of times the operation has been retried
 * @return the server response
 */
private CompletionStage<Response> retry(Supplier<CompletionStage<Response>> request, Executor executor, int tries)
{
    if (tries >= MAX_RETRIES)
        throw new CompletionException(new IOException("Request failed after " + MAX_RETRIES + " tries"));
    return request.get().thenComposeAsync(response ->
    {
        if (response.getStatusInfo().getFamily() != Response.Status.Family.SUCCESSFUL)
            return retry(request, executor, tries + 1);
        return CompletableFuture.completedFuture(response);
    }, executor);
}

Answer 7

也許為時已晚，但希望有人可能會覺得這很有用，我最近解決了這個問題，以便在失敗時重試 rest API 調用。 就我而言，我必須重試 500 HTTP 狀態代碼，下面是我的其余客戶端代碼（我們使用的是播放框架中的 WSClient），您可以根據需要將其更改為任何其余客戶端。

 int MAX_RETRY = 3;
 CompletableFuture<WSResponse> future = new CompletableFuture<>();

 private CompletionStage<WSResponse> getWS(Object request,String url, int retry, CompletableFuture future) throws JsonProcessingException {
 ws.url(url)
        .post(Json.parse(mapper.writeValueAsString(request)))
        .whenCompleteAsync((wsResponse, exception) -> {
            if(wsResponse.getStatus() == 500 && retry < MAX_RETRY) {
                try {
                    getWS(request, retry+1, future);
                } catch (IOException e) {
                    throw new Exception(e);
                }
            }else {
                future.complete(wsResponse);
            }
        });

     return future;
}

如果狀態代碼是 200 或 500 以外的，此代碼將立即返回，而如果 HTTP 狀態是 500，它將重試 3 次。

Answer 8

我們需要根據錯誤情況重試任務。

public static <T> CompletableFuture<T> retryOnCondition(Supplier<CompletableFuture<T>> supplier,
                                             Predicate<Throwable> retryPredicate, int maxAttempts) {
    if (maxAttempts <= 0) {
        throw new IllegalArgumentException("maxAttempts can't be <= 0");
    }
    return retryOnCondition(supplier, retryPredicate, null, maxAttempts);
}

private static <T> CompletableFuture<T> retryOnCondition(
    Supplier<CompletableFuture<T>> supplier, Predicate<Throwable> retryPredicate,
    Throwable lastError, int attemptsLeft) {

    if (attemptsLeft == 0) {
        return CompletableFuture.failedFuture(lastError);
    }

    return supplier.get()
        .thenApply(CompletableFuture::completedFuture)
        .exceptionally(error -> {
            boolean doRetry = retryPredicate.test(error);
            int attempts = doRetry ? attemptsLeft - 1 : 0;
            return retryOnCondition(supplier, retryPredicate, error, attempts);
        })
        .thenCompose(Function.identity());
}

用法：

public static void main(String[] args) {
    retryOnCondition(() -> myTask(), e -> {
        //log exception
        return e instanceof MyException;
    }, 3).join();
}

Answer 9

我建議在此用例中使用 resilience4j。 這非常方便！

簡介： resilience4j-retry及其 Javadoc： Retry

他們有直接裝飾 completionStage 的方法，如下所示：

default <T> java.util.concurrent.CompletionStage<T> executeCompletionStage​(java.util.concurrent.ScheduledExecutorService scheduler,
java.util.function.Supplier<java.util.concurrent.CompletionStage<T>> supplier)

Answer 10

靈感來自theazureshadow的回答。 他或她的回答很好，但不適用於新版本的 FailSafe。 下面的代碼適用於

<dependency>
  <groupId>dev.failsafe</groupId>
  <artifactId>failsafe</artifactId>
  <version>3.3.0</version>
</dependency>

解決方案：

    RetryPolicy<Object> retryPolicy = RetryPolicy.builder()
        .withMaxRetries(MAX_RETRY)
        .withBackoff(INITIAL_DELAY, MAX_DELAY, ChronoUnit.SECONDS)
        .build();
    Fallback<Object> fallback = Fallback.of((AuditEvent) null);

    public CompletableFuture<Object> executeAsync(Runnable asyncTask) {
    return Failsafe.with(fallback)
            .compose(retryPolicy)
            .with(executorService)
            .onFailure(e -> LOG.error(e.getException().getMessage()))
            .getAsync(() -> asyncTask());
}