Integrate HTTP request/response with asynchronous messages in RabbitMQ

Question

Our app is a message processing system with multiple components connected with RabbitMQ queues. So the message processing are asynchronous. Now I need to add a HTTP adapter that communicates with the system. Since HTTP is synchronous with request/response, I need a way to connect synchronous and asynchronous flows. The current solution is:

1. HTTP requests are sent to one queue. Each request has a unique request ID for correlation.
2. HTTP request is blocked by a CompletableFuture .
3. The request is processed and response is sent back to another queue.
4. Queue consumer uses response to complete the CompletableFuture matching with request ID.

The HTTP adapter is implemented using Akka HTTP. Requests are handled using handleWithAsyncHandler() with a function of type Function<HttpRequest, CompletionStage<HttpResponse>> .

The problem is that the HTTP adapter needs to manage a map ( Map<String, CompletableFuture> ) of all pending requests. For each request, a new CompletableFuture object is created and put into the map. When a response is received in the queue, the matching CompletableFuture is completed to finish the request. This seems a bad smell in the code because I need to carefully manage this map. For example, if a response failed to generate for a request, the request needs to be removed from the map.

I wonder if there are other ways than using a map to track all pending requests.

Answer 1

Basically, akka-http could be async style. You do not need to implement that queue to map the request Id.

One thing needs to be considered that DO NOT use the default dispatcher .

Better to define a blocking dispatcher to handle CompletableFuture.supplyAsync

For example

my-blocking-dispatcher {
  type = Dispatcher
  executor = "thread-pool-executor"
  thread-pool-executor {
    fixed-pool-size = 16
  }
  throughput = 1
}

import static akka.http.javadsl.server.Directives.completeWithFuture;
import static akka.http.javadsl.server.Directives.post;

// GOOD (the blocking is now isolated onto a dedicated dispatcher):
final Route routes = post(() -> {
    final MessageDispatcher dispatcher = system.dispatchers().lookup("my-blocking-dispatcher");
    return completeWithFuture(CompletableFuture.supplyAsync(() -> {
                try {
                    Thread.sleep(5000L);
                } catch (InterruptedException e) {
                }
                return HttpResponse.create()
                        .withEntity(Long.toString(System.currentTimeMillis()));
            }, dispatcher // uses the good "blocking dispatcher" that we
            // configured, instead of the default dispatcher to isolate the blocking.
    ));
});