org.apache.http.impl.execchain.RequestAbortedException: Request aborted

Question

I'm running into an issue when using apache http client for making http calls to a third part system from my web application. I'm using http-core-4.4.4 and http-client-4.5.1. This ticket HTTPCORE-446 indicates that the dead lock issue is resolved and terminate the thread gracefully with exception. Has anyone faced this exception and, any idea on what is causing this issue and how to resolve this?.

EDIT: Http code snippet used to make http calls

 protected <ResponseObj> run() throws Exception {

    String uri = <target system uri>;
    HttpPost httpPost = new HttpPost(uri);
    final HttpEntity entity = <Built http entity obj>;
    httpPost.setEntity(entity);
    HttpClient client = HttpClientBuilder.create().build();
    try {
        final HttpResponse response = client.execute(httpPost);
        ResponseObj res = processResponse(response);

        return res;

    } catch (Exception e) {

        LOGGER.error(ERROR_CALLING_TARGET, e);
        throw new CustomException(e);
    }
}

Exception

org.apache.http.impl.execchain.RequestAbortedException: Request aborted
at     org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:193) ~[httpclient-4.5.1.jar:4.5.1]
at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:184) ~[httpclient-4.5.1.jar:4.5.1]
at org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:88) ~[httpclient-4.5.1.jar:4.5.1]
at org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:110) ~[httpclient-4.5.1.jar:4.5.1]
at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:184) ~[httpclient-4.5.1.jar:4.5.1]
at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:82) ~[httpclient-4.5.1.jar:4.5.1]
at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:107) ~[httpclient-4.5.1.jar:4.5.1]
at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:55) ~[httpclient-4.5.1.jar:4.5.1]
at com.netflix.hystrix.HystrixCommand$1.call(HystrixCommand.java:294) [hystrix-core-1.4.26.jar:1.4.26]
at com.netflix.hystrix.HystrixCommand$1.call(HystrixCommand.java:289) [hystrix-core-1.4.26.jar:1.4.26]
at rx.Observable$2.call(Observable.java:162) [rxjava-1.1.1.jar:1.1.1]
at rx.Observable$2.call(Observable.java:154) [rxjava-1.1.1.jar:1.1.1]
at rx.Observable$2.call(Observable.java:162) [rxjava-1.1.1.jar:1.1.1]
at rx.Observable$2.call(Observable.java:154) [rxjava-1.1.1.jar:1.1.1]
at rx.Observable$2.call(Observable.java:162) [rxjava-1.1.1.jar:1.1.1]
at rx.Observable$2.call(Observable.java:154) [rxjava-1.1.1.jar:1.1.1]
at rx.Observable.unsafeSubscribe(Observable.java:8314) [rxjava-1.1.1.jar:1.1.1]
at com.netflix.hystrix.AbstractCommand$5.call(AbstractCommand.java:521) [hystrix-core-1.4.26.jar:1.4.26]
at com.netflix.hystrix.AbstractCommand$5.call(AbstractCommand.java:499) [hystrix-core-1.4.26.jar:1.4.26]
at rx.Observable.unsafeSubscribe(Observable.java:8314) [rxjava-1.1.1.jar:1.1.1]
at rx.internal.operators.OperatorSubscribeOn$1.call(OperatorSubscribeOn.java:94) [rxjava-1.1.1.jar:1.1.1]
at com.netflix.hystrix.strategy.concurrency.HystrixContexSchedulerAction$1.call(HystrixContexSchedulerAction.java:56) [hystrix-core-1.4.26.jar:1.4.26]
at com.netflix.hystrix.strategy.concurrency.HystrixContexSchedulerAction$1.call(HystrixContexSchedulerAction.java:47) [hystrix-core-1.4.26.jar:1.4.26]
at com.netflix.hystrix.strategy.concurrency.HystrixContexSchedulerAction.call(HystrixContexSchedulerAction.java:69) [hystrix-core-1.4.26.jar:1.4.26]
at rx.internal.schedulers.ScheduledAction.run(ScheduledAction.java:55) [rxjava-1.1.1.jar:1.1.1]
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) [na:1.8.0_51]
at java.util.concurrent.FutureTask.run(FutureTask.java:266) [na:1.8.0_51]
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) [na:1.8.0_51]
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) [na:1.8.0_51]
at java.lang.Thread.null(Unknown Source) [na:1.8.0_51]
Caused by: java.lang.InterruptedException: null
at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.reportInterruptAfterWait(AbstractQueuedSynchronizer.java:2014) ~[na:1.8.0_51]
at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2048) ~[na:1.8.0_51]
at org.apache.http.pool.PoolEntryFuture.await(PoolEntryFuture.java:138) ~[httpcore-4.4.4.jar:4.4.4]
at org.apache.http.pool.AbstractConnPool.getPoolEntryBlocking(AbstractConnPool.java:306) ~[httpcore-4.4.4.jar:4.4.4]
at org.apache.http.pool.AbstractConnPool.access$000(AbstractConnPool.java:64) ~[httpcore-4.4.4.jar:4.4.4]
at org.apache.http.pool.AbstractConnPool$2.getPoolEntry(AbstractConnPool.java:192) ~[httpcore-4.4.4.jar:4.4.4]
at org.apache.http.pool.AbstractConnPool$2.getPoolEntry(AbstractConnPool.java:185) ~[httpcore-4.4.4.jar:4.4.4]
at org.apache.http.pool.PoolEntryFuture.get(PoolEntryFuture.java:107) ~[httpcore-4.4.4.jar:4.4.4]
at org.apache.http.impl.conn.PoolingHttpClientConnectionManager.leaseConnection(PoolingHttpClientConnectionManager.java:276) ~[httpclient-4.5.1.jar:4.5.1]
at org.apache.http.impl.conn.PoolingHttpClientConnectionManager$1.get(PoolingHttpClientConnectionManager.java:263) ~[httpclient-4.5.1.jar:4.5.1]
at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:190) ~[httpclient-4.5.1.jar:4.5.1]
... 32 common frames omitted

Answer 1

I just encountered a similar problem, and I guess I can give some hints solving yours, too.

I had a HttpClient periodically calling a server, which proved to be very slow answering the calls. Request N was still waiting for a response when the scheduler started request N+1.

If you look at the source of MainClientExec ln190 shown at the bottom of your stacktrace, you can see that a ConnectionRequest.get() call throwed the original InterruptedException which was catched, wrapped into a RequestAbortedException, and rethrown. Quote from ConnectionRequest.get() methods javadoc:

Obtains a connection within a given time. This method will block until a connection becomes available, the timeout expires, or the connection manager is shut down. Timeouts are handled with millisecond precision. If Cancellable.cancel() is called while this is blocking or before this began, an InterruptedException will be thrown.

So, what is the thing that calls cancel() on our thread? To me, it looks like it's another thread executing the same MainClientExec.execute() method, likely with a request set up the same way. Just a few lines above the try-catch block:

final ConnectionRequest connRequest = connManager.requestConnection(route, userToken);
if (execAware != null) {
    if (execAware.isAborted()) {
        connRequest.cancel();
        throw new RequestAbortedException("Request aborted");
    } else {
        execAware.setCancellable(connRequest);
    }
}

My guess is that this code cancels the previous HTTP call if it is still in progress.

So, in my case the root problem was a slow server (took minutes to respond). My solution was to set connection and request timeout values. By default, no timeouts are set for HttpClient, you have to set it manually. See this answer for a working example on how to do this. Now my HTTP requests run into a timeout before the next scheduled call is triggered. I still have errors when the server is slow to respond, but timeout exceptions are a lot easier to grok.