具有客户端证书身份验证的HttpClient后请求

Question

A little background :

I'm trying to send test result files to the eVpT . That is done via a GKV server: https://verarbeitung.gkv-kommunikationsserver.de/meldung/extra14.meldung

We have a similar setup for the production counterpart and it sends to the same GKV server.

To the problem :

First I generated a keypair, something like this:

keytool -genkeypair -v -alias "TEST" -validity 1095 -keyalg RSA -sigalg SHA256withRSA -keysize 2048 -keystore TEST.keystore -dname "CN=NAME, OU=BN1234, OU=COMPANY NAME, O=ITSG TrustCenter fuer Arbeitgeber, C=DE"

With the resulting keystore file I created a certificate request:

keytool -certreq -alias "TEST" -file TEST.p10 -sigalg SHA256withRSA -keystore TEST.keystore

This was sent to the GKV server for signing and back came a .p7c file that I imported to the aforementioned keystore file:

keytool -import -keystore TEST.keystore -alias TEST -trustcacerts -file TEST.p7c

THe keystore type is JKS if that matters.

The setup for the request is as follow:

public HttpResponse send(String request, String serverUrl, boolean isEvptTest) {
    HttpClient httpClient = new HttpClient();
    return httpClient.post(request, serverUrl, new HttpClient.HttpPostCallback() {
        @Override
        public void configure(DefaultHttpClient defaultHttpClient, HttpPost httpPost) {
            try {
                httpPost.setHeader("Content-Type", "application/octet-stream");
                registerHttpsScheme(defaultHttpClient);
                addProxy(defaultHttpClient);
            } catch (Exception e) {
                throw new RuntimeException("could not register https scheme", e);
            }
        }
    });
}

private void addProxy(DefaultHttpClient httpClient) {
    String proxyHost = System.getProperty(Constants.HTTP_PROXY_HOST_PROPERTY);
    if (!Util.isEmpty(proxyHost)) {
        String proxyPort = System.getProperty(Constants.HTTP_PROXY_PORT_PROPERTY);
        HttpHost proxy = new HttpHost(proxyHost, Integer.parseInt(proxyPort), "http");
        httpClient.getParams().setParameter(ConnRoutePNames.DEFAULT_PROXY, proxy);
        log.debug(String.format("added proxy host %s port %s", proxyHost, proxyPort));
    }
}

private void registerHttpsScheme(DefaultHttpClient httpClient) throws NoSuchAlgorithmException, KeyManagementException, KeyStoreException, UnrecoverableKeyException {
    SSLSocketFactory socketFactory = new SSLSocketFactory(testKeyStore, testKeyStoreService.getKeyStorePassword(), trustStore);
    port = 443;
    Scheme sch = new Scheme("https", port, socketFactory);
    httpClient.getConnectionManager().getSchemeRegistry().register(sch);
}

The post method in the HttpClient:

public HttpResponse post(String content, String url, HttpPostCallback callback) {
    DefaultHttpClient httpClient = null;
    org.apache.http.HttpResponse response = null;
    HttpResponse serverResponse = new HttpResponse();
    try {
        httpClient = new DefaultHttpClient();
        HttpPost httpPost = new HttpPost(url);
        httpPost.setEntity(new StringEntity(content));

        if (callback != null) {
            callback.configure(httpClient, httpPost);
        }

        if (log.isDebugEnabled()) {
            log.debug(format("will post to url[%s] the following content[%s]", url, content));
            Header[] allHeaders = httpPost.getAllHeaders();
            for (Header header : allHeaders) {
                log.debug("header = " + header);
            }
        }

        response = httpClient.execute(httpPost);
        serverResponse.setStatusCode(response.getStatusLine().getStatusCode());
        log.debug(format("server response statuscode[%s]", serverResponse.getStatusCode()));
        String entity = EntityUtils.toString(response.getEntity());
        serverResponse.setResponse(entity);
        if (log.isDebugEnabled()) {
            log.debug(format("received response [%s]", entity));
        }
    } catch (Exception e) {
        log.error(format("error while sending to [%s]", url), e);
        serverResponse.setResponse(e.getMessage());
    } finally {
        consumeContent(httpClient, response);
    }
    return serverResponse;
}

this works for normal transactions with the GKV server, but when I try with the test keystore all I get is a 403 response.

---

I have run my program with the proposed javax.net.debug flag and got the following (if any part of the needs more detail please let me know):

found key for : *1234*
chain [0] = [
[
  Version: V3
  Subject: CN=*NAME*, OU=*BN1234*, OU=*COMPANY NAME*, O=ITSG TrustCenter fuer Arbeitgeber, C=DE
  Signature Algorithm: SHA256withRSA, OID = *SOMEID*
...

then comes a list of all the trusted certs being added including:

*.gkv-kommunikationsserver.de

When the actual http POST request is being made:

http-bio-8080-exec-4, setSoTimeout(0) called
Allow unsafe renegotiation: false
Allow legacy hello messages: true
Is initial handshake: true
Is secure renegotiation: false
Ignoring unsupported cipher suite: TLS_ECDHE_ECDSA_WITH_AES_256_CBC_SHA384 for TLSv1
Ignoring unsupported cipher suite: TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384 for TLSv1
Ignoring unsupported cipher suite: TLS_RSA_WITH_AES_256_CBC_SHA256 for TLSv1
Ignoring unsupported cipher suite: TLS_ECDH_ECDSA_WITH_AES_256_CBC_SHA384 for TLSv1
Ignoring unsupported cipher suite: TLS_ECDH_RSA_WITH_AES_256_CBC_SHA384 for TLSv1
Ignoring unsupported cipher suite: TLS_DHE_RSA_WITH_AES_256_CBC_SHA256 for TLSv1
Ignoring unsupported cipher suite: TLS_DHE_DSS_WITH_AES_256_CBC_SHA256 for TLSv1
Ignoring unsupported cipher suite: TLS_ECDHE_ECDSA_WITH_AES_256_CBC_SHA384 for TLSv1.1
Ignoring unsupported cipher suite: TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384 for TLSv1.1
Ignoring unsupported cipher suite: TLS_RSA_WITH_AES_256_CBC_SHA256 for TLSv1.1
Ignoring unsupported cipher suite: TLS_ECDH_ECDSA_WITH_AES_256_CBC_SHA384 for TLSv1.1
Ignoring unsupported cipher suite: TLS_ECDH_RSA_WITH_AES_256_CBC_SHA384 for TLSv1.1
Ignoring unsupported cipher suite: TLS_DHE_RSA_WITH_AES_256_CBC_SHA256 for TLSv1.1
Ignoring unsupported cipher suite: TLS_DHE_DSS_WITH_AES_256_CBC_SHA256 for TLSv1.1
%% No cached client session
*** ClientHello, TLSv1.2
RandomCookie:  GMT: 1484796190 bytes = { 224, 120, 124, 66, 118, 48, 83, 225, 119, 151, 135, 164, 71, 96, 51, 216, 132, 136, 223, 123, 66, 151, 254, 6, 198, 88, 86, 67 }
Session ID:  {}
Cipher Suites: [TLS_ECDHE_ECDSA_WITH_AES_256_CBC_SHA384, TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384, TLS_RSA_WITH_AES_256_CBC_SHA256, TLS_ECDH_ECDSA_WITH_AES_256_CBC_SHA384, TLS_ECDH_RSA_WITH_AES_256_CBC_SHA384, TLS_DHE_RSA_WITH_AES_256_CBC_SHA256, TLS_DHE_DSS_WITH_AES_256_CBC_SHA256, TLS_ECDHE_ECDSA_WITH_AES_256_CBC_SHA, TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA, TLS_RSA_WITH_AES_256_CBC_SHA, TLS_ECDH_ECDSA_WITH_AES_256_CBC_SHA, TLS_ECDH_RSA_WITH_AES_256_CBC_SHA, TLS_DHE_RSA_WITH_AES_256_CBC_SHA, TLS_DHE_DSS_WITH_AES_256_CBC_SHA, TLS_ECDHE_ECDSA_WITH_AES_128_CBC_SHA256, TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA256, TLS_RSA_WITH_AES_128_CBC_SHA256, TLS_ECDH_ECDSA_WITH_AES_128_CBC_SHA256, TLS_ECDH_RSA_WITH_AES_128_CBC_SHA256, TLS_DHE_RSA_WITH_AES_128_CBC_SHA256, TLS_DHE_DSS_WITH_AES_128_CBC_SHA256, TLS_ECDHE_ECDSA_WITH_AES_128_CBC_SHA, TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA, TLS_RSA_WITH_AES_128_CBC_SHA, TLS_ECDH_ECDSA_WITH_AES_128_CBC_SHA, TLS_ECDH_RSA_WITH_AES_128_CBC_SHA, TLS_DHE_RSA_WITH_AES_128_CBC_SHA, TLS_DHE_DSS_WITH_AES_128_CBC_SHA, TLS_ECDHE_ECDSA_WITH_AES_256_GCM_SHA384, TLS_ECDHE_ECDSA_WITH_AES_128_GCM_SHA256, TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384, TLS_RSA_WITH_AES_256_GCM_SHA384, TLS_ECDH_ECDSA_WITH_AES_256_GCM_SHA384, TLS_ECDH_RSA_WITH_AES_256_GCM_SHA384, TLS_DHE_RSA_WITH_AES_256_GCM_SHA384, TLS_DHE_DSS_WITH_AES_256_GCM_SHA384, TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256, TLS_RSA_WITH_AES_128_GCM_SHA256, TLS_ECDH_ECDSA_WITH_AES_128_GCM_SHA256, TLS_ECDH_RSA_WITH_AES_128_GCM_SHA256, TLS_DHE_RSA_WITH_AES_128_GCM_SHA256, TLS_DHE_DSS_WITH_AES_128_GCM_SHA256, TLS_ECDHE_ECDSA_WITH_3DES_EDE_CBC_SHA, TLS_ECDHE_RSA_WITH_3DES_EDE_CBC_SHA, SSL_RSA_WITH_3DES_EDE_CBC_SHA, TLS_ECDH_ECDSA_WITH_3DES_EDE_CBC_SHA, TLS_ECDH_RSA_WITH_3DES_EDE_CBC_SHA, SSL_DHE_RSA_WITH_3DES_EDE_CBC_SHA, SSL_DHE_DSS_WITH_3DES_EDE_CBC_SHA, TLS_EMPTY_RENEGOTIATION_INFO_SCSV]
Compression Methods:  { 0 }
Extension elliptic_curves, curve names: {secp256r1, sect163k1, sect163r2, secp192r1, secp224r1, sect233k1, sect233r1, sect283k1, sect283r1, secp384r1, sect409k1, sect409r1, secp521r1, sect571k1, sect571r1, secp160k1, secp160r1, secp160r2, sect163r1, secp192k1, sect193r1, sect193r2, secp224k1, sect239k1, secp256k1}
Extension ec_point_formats, formats: [uncompressed]
Extension signature_algorithms, signature_algorithms: SHA512withECDSA, SHA512withRSA, SHA384withECDSA, SHA384withRSA, SHA256withECDSA, SHA256withRSA, SHA1withECDSA, SHA1withRSA, SHA1withDSA
***

...

http-bio-8080-exec-4, READ: TLSv1.2 Handshake, length = 3043
*** ServerHello, TLSv1.2
RandomCookie:  GMT: 1484796191 bytes = { 74, 209, 14, 128, 123, 109, 178, 53, 206, 74, 209, 82, 66, 241, 252, 13, 234, 164, 2, 171, 46, 102, 203, 234, 224, 28, 213, 70 }
Session ID:  {154, 66, 0, 0, 252, 124, 28, 13, 62, 10, 104, 127, 66, 84, 189, 234, 174, 203, 170, 168, 194, 0, 146, 12, 98, 164, 135, 5, 176, 186, 95, 180}
Cipher Suite: TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384
Compression Method: 0
Extension renegotiation_info, renegotiated_connection: <empty>
***

...

*** Certificate chain
chain [0] = [
[
  Version: V3
  Subject: CN=*.gkv-kommunikationsserver.de, O=ITSG GmbH, OU=Systeme und Netze, L=Heusenstamm, ST=Hessen, C=DE
  Signature Algorithm: SHA256withRSA, OID = 1.2.840.113549.1.1.11

...

***
Found trusted certificate:
[
[
  Version: V3
  Subject: CN=*.gkv-kommunikationsserver.de, O=ITSG GmbH, OU=Systeme und Netze, L=Heusenstamm, ST=Hessen, C=DE
  Signature Algorithm: SHA256withRSA, OID = 1.2.840.113549.1.1.11

...

*** ECDH ServerKeyExchange
Signature Algorithm SHA1withRSA
Server key: Sun EC public key, 521 bits

...

*** ECDHClientKeyExchange
SESSION KEYGEN:
PreMaster Secret:
CONNECTION KEYGEN:
Client Nonce:
Server Nonce:
Master Secret:
Client MAC write Secret:
Server MAC write Secret:
Client write key:
Server write key:
... no IV derived for this protocol
http-bio-8080-exec-4, WRITE: TLSv1.2 Change Cipher Spec, length = 1

... Then it begins to verify the data which seems to go just fine. Then for some reason a new Hello request is made:

*** HelloRequest (empty)
%% Client cached [Session-1, TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384]
%% Try resuming [Session-1, TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384] from port 62362
*** ClientHello, TLSv1.2
RandomCookie:  GMT: 1484796208 bytes = { 130, 109, 91, 183, 192, 75, 115, 223, 4, 85, 231, 11, 50, 168, 37, 5, 25, 155, 187, 103, 246, 21, 104, 162, 129, 137, 11, 19 }
Session ID:  {154, 66, 0, 0, 252, 124, 28, 13, 62, 10, 104, 127, 66, 84, 189, 234, 174, 203, 170, 168, 194, 0, 146, 12, 98, 164, 135, 5, 176, 186, 95, 180}
Cipher Suites: [TLS_ECDHE_ECDSA_WITH_AES_256_CBC_SHA384, TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384, TLS_RSA_WITH_AES_256_CBC_SHA256, TLS_ECDH_ECDSA_WITH_AES_256_CBC_SHA384, TLS_ECDH_RSA_WITH_AES_256_CBC_SHA384, TLS_DHE_RSA_WITH_AES_256_CBC_SHA256, TLS_DHE_DSS_WITH_AES_256_CBC_SHA256, TLS_ECDHE_ECDSA_WITH_AES_256_CBC_SHA, TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA, TLS_RSA_WITH_AES_256_CBC_SHA, TLS_ECDH_ECDSA_WITH_AES_256_CBC_SHA, TLS_ECDH_RSA_WITH_AES_256_CBC_SHA, TLS_DHE_RSA_WITH_AES_256_CBC_SHA, TLS_DHE_DSS_WITH_AES_256_CBC_SHA, TLS_ECDHE_ECDSA_WITH_AES_128_CBC_SHA256, TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA256, TLS_RSA_WITH_AES_128_CBC_SHA256, TLS_ECDH_ECDSA_WITH_AES_128_CBC_SHA256, TLS_ECDH_RSA_WITH_AES_128_CBC_SHA256, TLS_DHE_RSA_WITH_AES_128_CBC_SHA256, TLS_DHE_DSS_WITH_AES_128_CBC_SHA256, TLS_ECDHE_ECDSA_WITH_AES_128_CBC_SHA, TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA, TLS_RSA_WITH_AES_128_CBC_SHA, TLS_ECDH_ECDSA_WITH_AES_128_CBC_SHA, TLS_ECDH_RSA_WITH_AES_128_CBC_SHA, TLS_DHE_RSA_WITH_AES_128_CBC_SHA, TLS_DHE_DSS_WITH_AES_128_CBC_SHA, TLS_ECDHE_ECDSA_WITH_AES_256_GCM_SHA384, TLS_ECDHE_ECDSA_WITH_AES_128_GCM_SHA256, TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384, TLS_RSA_WITH_AES_256_GCM_SHA384, TLS_ECDH_ECDSA_WITH_AES_256_GCM_SHA384, TLS_ECDH_RSA_WITH_AES_256_GCM_SHA384, TLS_DHE_RSA_WITH_AES_256_GCM_SHA384, TLS_DHE_DSS_WITH_AES_256_GCM_SHA384, TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256, TLS_RSA_WITH_AES_128_GCM_SHA256, TLS_ECDH_ECDSA_WITH_AES_128_GCM_SHA256, TLS_ECDH_RSA_WITH_AES_128_GCM_SHA256, TLS_DHE_RSA_WITH_AES_128_GCM_SHA256, TLS_DHE_DSS_WITH_AES_128_GCM_SHA256, TLS_ECDHE_ECDSA_WITH_3DES_EDE_CBC_SHA, TLS_ECDHE_RSA_WITH_3DES_EDE_CBC_SHA, SSL_RSA_WITH_3DES_EDE_CBC_SHA, TLS_ECDH_ECDSA_WITH_3DES_EDE_CBC_SHA, TLS_ECDH_RSA_WITH_3DES_EDE_CBC_SHA, SSL_DHE_RSA_WITH_3DES_EDE_CBC_SHA, SSL_DHE_DSS_WITH_3DES_EDE_CBC_SHA]
Compression Methods:  { 0 }
Extension elliptic_curves, curve names: {secp256r1, sect163k1, sect163r2, secp192r1, secp224r1, sect233k1, sect233r1, sect283k1, sect283r1, secp384r1, sect409k1, sect409r1, secp521r1, sect571k1, sect571r1, secp160k1, secp160r1, secp160r2, sect163r1, secp192k1, sect193r1, sect193r2, secp224k1, sect239k1, secp256k1}
Extension ec_point_formats, formats: [uncompressed]
Extension signature_algorithms, signature_algorithms: SHA512withECDSA, SHA512withRSA, SHA384withECDSA, SHA384withRSA, SHA256withECDSA, SHA256withRSA, SHA1withECDSA, SHA1withRSA, SHA1withDSA
Extension renegotiation_info, renegotiated_connection: 39:a7:89:65:54:45:31:c8:c1:bc:93:b0
***

...

*** ServerHello, TLSv1.2
RandomCookie:  GMT: 1484796208 bytes = { 104, 194, 244, 49, 210, 153, 252, 205, 241, 25, 177, 239, 212, 114, 206, 70, 186, 6, 152, 31, 243, 128, 145, 3, 86, 42, 208, 145 }
Session ID:  {158, 32, 0, 0, 233, 255, 55, 252, 77, 3, 17, 171, 9, 23, 239, 78, 0, 229, 23, 91, 244, 92, 163, 129, 33, 72, 196, 250, 199, 121, 51, 85}
Cipher Suite: TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384
Compression Method: 0
Extension renegotiation_info, renegotiated_connection: 39:a7:89:65:54:45:31:c8:c1:bc:93:b0:f1:24:0e:aa:fc:54:f3:7a:1a:79:26:8e
***

...

*** CertificateRequest
Cert Types: RSA, DSS, ECDSA
Supported Signature Algorithms: SHA512withRSA, SHA512withECDSA, SHA256withRSA, SHA384withRSA, SHA1withRSA, SHA256withECDSA, SHA384withECDSA, SHA1withECDSA, SHA1withDSA
Cert Authorities:
<CN=AddTrust External CA Root, OU=AddTrust External TTP Network, O=AddTrust AB, C=SE>
<CN=StartCom Certification Authority, OU=Secure Digital Certificate Signing, O=StartCom Ltd., C=IL>
<CN=VeriSign Class 3 Public Primary Certification Authority - G5, OU="(c) 2006 VeriSign, Inc. - For authorized use only", OU=VeriSign Trust Network, O="VeriSign, Inc.", C=US>
<OU=Class 3 Public Primary Certification Authority, O="VeriSign, Inc.", C=US>
<CN=GlobalSign Root CA, OU=Root CA, O=GlobalSign nv-sa, C=BE>
<CN=Baltimore CyberTrust Root, OU=CyberTrust, O=Baltimore, C=IE>
<CN=Microsoft Root Certificate Authority 2010, O=Microsoft Corporation, L=Redmond, ST=Washington, C=US>
<O=Datenaustausch im Gesundheits- und Sozialwesen, C=DE>
<CN=Microsoft Root Certificate Authority 2011, O=Microsoft Corporation, L=Redmond, ST=Washington, C=US>
<CN=Microsoft Root Authority, OU=Microsoft Corporation, OU=Copyright (c) 1997 Microsoft Corp.>
<CN=Microsoft Root Certificate Authority, DC=microsoft, DC=com>
[read] MD5 and SHA1 hashes:  len = 1336

...

*** ServerHelloDone
[read] MD5 and SHA1 hashes:  len = 4
0000: 0E 00 00 00                                        ....
Warning: no suitable certificate found - continuing without client authentication
*** Certificate chain

... Is it just me or does this tell me that my client certificate has not been added to the server truststore?

Answer 1

Couple of things to try:

1. Is the root of the CA issuance chain for your client cert in your trust store? If not, add that otherwise it will not reconcile where your certificate was issued from and will not select it.
2. You can run your program with -Djavax.net.debug=all . This will dump a metric ... tonne of information to your console. It will show all of the certificates loaded as trusted issuers, the handshake parameters, as well as the interaction between the server and client on determining who has to provide what in the exchange.

---

After a protracted discussion we got to the root of the issue. Ricco had just about everything necessary in place except for the root certificate of the issuance chain in the trust store.

In the mechanics of the TLS handshake, the server presented a list of issuance authorities

*** CertificateRequest
Cert Types: RSA, DSS, ECDSA
Supported Signature Algorithms: SHA512withRSA, SHA512withECDSA, SHA256withRSA, SHA384withRSA, SHA1withRSA, SHA256withECDSA, SHA384withECDSA, SHA1withECDSA, SHA1withDSA
Cert Authorities:
<CN=AddTrust External CA Root, OU=AddTrust External TTP Network, O=AddTrust AB, C=SE>
<CN=StartCom Certification Authority, OU=Secure Digital Certificate Signing, O=StartCom Ltd., C=IL>
<CN=VeriSign Class 3 Public Primary Certification Authority - G5, OU="(c) 2006 VeriSign, Inc. - For authorized use only", OU=VeriSign Trust Network, O="VeriSign, Inc.", C=US>
<OU=Class 3 Public Primary Certification Authority, O="VeriSign, Inc.", C=US>
<CN=GlobalSign Root CA, OU=Root CA, O=GlobalSign nv-sa, C=BE>
<CN=Baltimore CyberTrust Root, OU=CyberTrust, O=Baltimore, C=IE>
<CN=Microsoft Root Certificate Authority 2010, O=Microsoft Corporation, L=Redmond, ST=Washington, C=US>
<O=Datenaustausch im Gesundheits- und Sozialwesen, C=DE>
<CN=Microsoft Root Certificate Authority 2011, O=Microsoft Corporation, L=Redmond, ST=Washington, C=US>
<CN=Microsoft Root Authority, OU=Microsoft Corporation, OU=Copyright (c) 1997 Microsoft Corp.>
<CN=Microsoft Root Certificate Authority, DC=microsoft, DC=com>

The trust engine will attempt to check to see if it knows about one of those issuance authorities by it's presence in the trust store. Once that is resolved, it will attempt to find the first privateKeyEntry in the keystore that has a matching certificate in it's chain and then, if the private key can be recovered, use that certificate to provide the second half of the authentication. See RFC 5246 for the gory details of the protocol handshake.

After adding the certificate associated with "<O=Datenaustausch im Gesundheits- und Sozialwesen, C=DE>" to the trust store Ricco was successful and got past the 403 error described in the question.

Also note, the server in this case was configured to "want" rather than "require" client certificate authentication. In "want" mode, the client certificate will be presented if one is available, but otherwise will operate in normal TLS connection mode. In "require" mode, the client certificate is required to be presented for the TLS connection to be completed.