Fastest way to retrieve Http response status in Java

Question

I'm doing a little research and benchmark in order to get the Status Code from an Http response.

I need to get response status code from a large list of sites using the fastest way possible, I can't run in multiple threads this will work in a batch process and it's a requisite.

I created a little benchmark service that sends request (avoiding site's cache) multiple times and retrieves time stats.

I'm testing with different clients and APIs but the response times aren't good enough (min 200ms, average: 300 - 400ms).

I test with Java URLConnection, SpringRestTemplate, ApacheHttpCommons, GoogleHttpClient and Restlet. One problem I've found is that in some of them I'cant request only the HEAD and then retrieve the status sode so I pick full response.

I'm also thinking in sockets and scripts.

Any help would be appreciated lot.

Answer 1

Lack of threading is a bigger problem that those slow response times (which are probably 99% network delay - the particular HTTP library is unlikely to make much difference).

You could try http://mina.apache.org/asyncweb/ which is built on Java NIO, and therefore nonblocking, but it might create threads in the background.