NTLM和Java Apache httpclient 4.5.6的多线程问题

Question

I have a stand-alone Java client trying to do RMI through a NTLM proxy. It's multithreaded. I'm using Apache httpclient 4.5.6. I've got the proxy on a 5 minute timeout cycle.

The basic case works, reauthenticating every 5 minutes when challenged by the proxy, as long as 2 threads don't make a request at the same time at exactly the time the proxy times out. Then it fails. Once it fails, all subsequent attempts fail.

I've attached a wireshark screenshot to clarify (screenshot is from 4.5.2 but I upgraded to 4.5.6 and saw the same behavior).

A good cycle looks like

• Client tries CONNECT (no NTML flags)
• Proxy replies with 407 (no NTML flags)
• Client tries CONNECT again with ntlm messagetype NTLMSSP_NEGOTIATE
• Proxy replies with 407 NTLMSSP_CHALLENGE
• Client does CONNECT with NTLMSSP_AUTH and my credentials.
• Proxy replies with 200, and we are good to go for another 5 minutes.

A bad cycle looks like

• Client tries CONNECT (no NTML flags)
• Proxy replies with 407 (no NTML flags)
• Client tries CONNECT again with ntlm messagetype NTLMSSP_NEGOTIATE
• Client tries CONNECT (no NTML flags)
• Proxy replies with 407 (no NTML flags)
• Proxy replies with 407 NTLMSSP_CHALLENGE
• A whole bunch more CONNECTs and 407s without NTML flags within a few seconds.

to me this looks like a multithread race condition in non-threadsafe code.

With Apache httpclient 4.5.2 it just propogated the 407 and I detected it in CloseableHttpResponse.getStatusLine().getStatusCode(). With Apache httpclient 4.5.6 I see this

java.lang.IllegalStateException: Auth scheme is null
    at org.apache.http.util.Asserts.notNull(Asserts.java:52)
    at org.apache.http.impl.auth.HttpAuthenticator.ensureAuthScheme(HttpAuthenticator.java:229)
    at org.apache.http.impl.auth.HttpAuthenticator.generateAuthResponse(HttpAuthenticator.java:184)
    at org.apache.http.impl.execchain.MainClientExec.createTunnelToTarget(MainClientExec.java:484)
    at org.apache.http.impl.execchain.MainClientExec.establishRoute(MainClientExec.java:411)
    at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:237)
    at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:185)
    at org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:89)
    at org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:110)
    at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:185)
    at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:83)

Any ideas how to protect against this or work around it or recover from it? (beside sync on the calls, which would slow down an already slow app a lot)

some code snippets from the app:

// this is done only once
HttpClientBuilder builder = HttpClients.custom();
SocketConfig.Builder socketConfig = SocketConfig.custom();
RequestConfig.Builder requestConfig = RequestConfig.custom();
HttpHost proxy = new HttpHost(proxyHost, proxyPort);
builder.setProxy(proxy);
requestConfig.setProxy(proxy);
builder.setProxyAuthenticationStrategy(new ProxyAuthenticationStrategy());
CredentialsProvider credentialsProvider = new BasicCredentialsProvider();
String localHost = getLocalHostname();
credentialsProvider.setCredentials(
    new AuthScope(proxyHost, proxyPort, AuthScope.ANY_REALM, "ntlm"),
    new NTCredentials(user, password, localHost, domain));
builder.setDefaultCredentialsProvider(credentialsProvider);
builder.setDefaultSocketConfig(socketConfig.build());
builder.setDefaultRequestConfig(requestConfig.build());
CloseableHttpClient client = builder.build();

...

// cached, we use the same one every time in accordance with section 4.7 of
// https://hc.apache.org/httpcomponents-client-4.5.x/tutorial/html/authentication.html
HttpClientContext context = HttpClientContext.create();
context.setCredentialsProvider(credentialsProvider);

...

// new HttpPost every time
HttpPost postMethod = new HttpPost(uri);
postMethod.setEntity(new ByteArrayEntity(bytesOut.toByteArray()));
response = client.execute(postMethod, context);

Answer 1

HttpContext instances are perfectly thread-safe. However some attributes stored in the context such as authentication handshake state are obviously not. Make sure that HttpContext instances do not get updated concurrently and the problem should go away.

Answer 2

Thank you Oleg, this is what I did and it seems to be working so far (too long to post as a comment on your answer, but I wanted to share my code)

// I use the base version when not going through a proxy
public class HttpClientContextFactory {
    public HttpClientContext create() {
        return HttpClientContext.create();
    }
}

// I use this when I go through a NTLM proxy
private HttpClientContextFactory getNtlmContextFactory(
        final CredentialsProvider credentialsProvider) {
    return new HttpClientContextFactory() {
        ThreadLocal<HttpClientContext> tlContext = ThreadLocal
                .<HttpClientContext> withInitial(() -> {
                    HttpClientContext context = HttpClientContext.create();
                    context.setCredentialsProvider(credentialsProvider);
                    return context;
                });

        @Override
        public HttpClientContext create() {
            return tlContext.get();
        }
    };
}

// then do this when I connect to the server
response = client.execute(postMethod, contextFactory.create());