通过toStrict（）将HttpEntity转换为String后导致延迟的原因

Question

I'm using akka-http for a web application that has a quite short response time target. I use the routing DSLs completeWithFuture method with a chain of CompletableFuture s.

I noticed that, when chaining every future using the XXXasync variant of the CompletionStage methods and passing the same executor, the thread used for processing the stage can change arbitrarily, causing a higher response time for some requests in case that all threads of the specified executor are used. So I passed my custom executor to the first CompletableFuture and chained all following stages with normal variant, in order to use the same thread for them.

Problem : One stage does the conversion of the HttpEntity to a String via HttpEntity.toStrict() and that method uses a thread from akka.actor.default-dispatcher . With increasing workload there are more and more requests that exceed the required response time at the beginning of the next stage, despite passing a timeout to toStrict that is way lower than the targeted response time and seeing no timeout exceptions.

Simplified Code :

private Route handleRequest(final HttpRequest request) {
    return completeWithFuture(CompletableFuture.runAsync(() -> preprocessing(), systemDispatcher) // Dispatcher 1
            .thenCompose((preprocessingResult) -> // please ignore that preprocessingResult is not used in that simplified version
            entityToString(request.entity()).thenApply((requestString) -> generateResponse(requestString))));
}

public CompletionStage<String> entityToString(final HttpEntity entity) {
        long start = System.nanoTime();
        return entity.toStrict(bodyReadTimeoutMillis, materializer).thenApplyAsync((final Strict strict) -> {
            System.out.println(start-System.nanoTime()); // varies between <1ms and >500ms
            return strict.getData().utf8String();
        }, systemDispatcher); // Dispatcher 2
    }

So my guess is that the switch from a thread of my custom executor to one of akkas default actor dispatcher thread and back is causing the problem.

Questions : Is there another explanation for the delay in my entityToString method? Is there a way to achieve the same as toStrict, ie getting the whole possibly chunked message body as string while avoiding to switch threads multiple times?

Please note that I need the timeout functionality of the toStrict method to abort processing of slow POST requests.

UPDATE

Thinking about it for the last days, I believe that it is not possible to achieve a non-blocking read, that akka guarantees, without switching threads. So the real problem is the noticeably high delay probably caused by the scheduling after toStrict .

I tried to use different dispatchers (see the comments Dispatcher 1 / Dispatcher 2 in the code above) and logged the inhabitants count of Dispatcher 2 in the case the delay exceeds 50ms. I can not find a proper documentation but I assume this is the number of scheduled tasks. I ran apache bench with 10000 requests, 200 concurrent connections and got 55 times a delay exceeding 50ms. The output shows a maximum of 80 inhabitants.

I ran that test on Amazons m3.2xlarge instance (8 vCPU, 30GB RAM, Ubuntu 16.04), no other processes consuming any noticeable amount of cpu). The dispatchers are of type fork-join-executor with parallelism-factor = 1.

Real traffic with a much more variable number of concurrent requests causes an increasing rate of requests that exceed the limit (up to 50%).

The average time for processing a request is below 1ms. What is causing that infrequent delay after toStrict and how to avoid it?

Answer 1

You can try get the entity content using akka-streams, and achieve timeout requirement in a different way instead of entity read.

entity.getDataBytes().runFold(ByteString.empty(), ByteString::concat, materializer)
.thenCompose(r -> r.utf8String());