通过异步http提高吞吐量

Question

In a java web-app I would like to maximize my server throughput with the following requirement:

• A system that gets multiple concurrent http requests.
• For each request some processing is required.
• Each request processing is consisted of a few steps.
• The last step calls an external API (http invocation).

Here is a basic diagram describing my system:

So for each request I'm:

• Creating 3 runnables and submitting (Using my executor).
• Then I'm waiting for all 3 to finish and keep on processing the results (doing it using Guava ListenableFuture )
• When done processing all results returning response

In this design I'm using a synchronous http client (when invoking the external services) meaning that each of my tasks is blocked once it gets to the 3rd step untill the http response arrives back. This could take up to a few hundreds of milliseconds.

My question is - I wonder if using an asynchronous http client can help me increase the throughput? With an asynchronous http client once I launch my http request the thread of the task is released back to the pool and available for additional processing and once the http response is back a new thread is assigned to continue the task. Does it make sense? If so, Does it depend upon the time it takes for the http external invocations to finish? If the response arrives back after 5ms will I still gain additional throughput? Has anyone measured something similar?

Answer 1

Assuming that by asynchronous client you do not mean the Servlet async context, but an implementation of HTTP client which supports Futures or Callbacks, you won't get any benefit from using asynchronous http client.

As you'll still have to wait on the Future returned by the HTTP client in the container thread responsible for request processing. However if you choose to use callback, I'm not sure how that callback will relaunch a container thread to return the response. It seems that correct way to do it with callback will be by using Servlet 3 async capability.

By using the Servlet async you'll be able to save time on the container thread and would be able to process more rquests in the same amount of time. Also in this case there will be no need to use a async HTTP client as HTTP API call is already in Servlet async thread.

It's unlikely that you'll gain throughput if the response (HTTP API call) gets back in 5ms. That said I haven't done any bench marking of my own for this and it would be interesting to see the results of such bench marking.