Https request with mutual authentication passes with curl but fails with java

Question

Someone on github asked me a question regarding my library. This library provides some factory classes to easily create a sslcontext. I make sure not the share the details of the library and just share plain java code and the additional library which I used. I tried to debug it and also tried reformatting of the certificates but it does not work. The exception which I got is:

sun.security.validator.ValidatorException: PKIX path building failed: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target

The java trustmanager is basically saying I received the certificate of the server and I just checked in your list of trusted certificates and I couldn't find a matching certificate. So I don't trust this server and therefor I am stopping this request. When I debugged it I could understand why the trustmanager throw an exception because there was no matching certificate at all which was the same as the server. However when I use the same files with curl it works, so I was not quite sure if there is an issue with the certificates and the key material, or maybe java ssl implementation has some limitations or maybe I am not configuring it correctly... I hope someone can explain me the difference in behaviour of why it fails with java and passes with curl.

The private key content of the client:

The certificate chain content of the client:

The trusted certificate, ca

So the following curl command will successfully execute and gives me the correct response body:

curl --cacert ca.pem --key client-key.pem --cert client-cert.pem https://prod.idrix.eu/secure/

To load these files in Java I use Bouncy Castle, below is the full snippet of parsing it, loading it into TrustManager, KeyManager, creating the SSLContext and executing the request:

import org.bouncycastle.asn1.pkcs.PrivateKeyInfo;
import org.bouncycastle.cert.X509CertificateHolder;
import org.bouncycastle.cert.jcajce.JcaX509CertificateConverter;
import org.bouncycastle.jce.provider.BouncyCastleProvider;
import org.bouncycastle.openssl.PEMKeyPair;
import org.bouncycastle.openssl.PEMParser;
import org.bouncycastle.openssl.jcajce.JcaPEMKeyConverter;

import javax.net.ssl.KeyManagerFactory;
import javax.net.ssl.SSLContext;
import javax.net.ssl.TrustManagerFactory;
import java.io.BufferedReader;
import java.io.IOException;
import java.io.InputStream;
import java.io.InputStreamReader;
import java.io.Reader;
import java.io.StringReader;
import java.net.URI;
import java.net.http.HttpClient;
import java.net.http.HttpRequest;
import java.net.http.HttpResponse;
import java.nio.charset.StandardCharsets;
import java.security.KeyStore;
import java.security.PrivateKey;
import java.security.cert.Certificate;
import java.security.cert.X509Certificate;
import java.util.Objects;
import java.util.stream.Collectors;

public class App {

    private static final BouncyCastleProvider BOUNCY_CASTLE_PROVIDER = new BouncyCastleProvider();
    private static final JcaPEMKeyConverter KEY_CONVERTER = new JcaPEMKeyConverter().setProvider(BOUNCY_CASTLE_PROVIDER);
    private static final JcaX509CertificateConverter CERTIFICATE_CONVERTER = new JcaX509CertificateConverter().setProvider(BOUNCY_CASTLE_PROVIDER);

    public static void main(String[] args) throws Exception {
        String clientKeyContent = App.getContent(App.class.getClassLoader().getResourceAsStream("client-key.pem"));
        String clientCertificateChainContent = App.getContent(App.class.getClassLoader().getResourceAsStream("client-cert.pem"));
        String caCertificateContent = App.getContent(App.class.getClassLoader().getResourceAsStream("ca.pem"));

        Reader reader = new StringReader(clientKeyContent);
        PEMParser pemParser = new PEMParser(reader);
        Object object = pemParser.readObject();

        PrivateKeyInfo privateKeyInfo = ((PEMKeyPair) object).getPrivateKeyInfo();
        PrivateKey clientKey = KEY_CONVERTER.getPrivateKey(privateKeyInfo);
        reader.close();
        pemParser.close();

        reader = new StringReader(clientCertificateChainContent);
        pemParser = new PEMParser(reader);

        object = pemParser.readObject();
        X509Certificate clientCertificateChain = CERTIFICATE_CONVERTER.getCertificate((X509CertificateHolder) object);
        reader.close();
        pemParser.close();

        reader = new StringReader(caCertificateContent);
        pemParser = new PEMParser(reader);

        object = pemParser.readObject();
        X509Certificate caCertificate = CERTIFICATE_CONVERTER.getCertificate((X509CertificateHolder) object);
        reader.close();
        pemParser.close();

        KeyStore keyStore = KeyStore.getInstance(KeyStore.getDefaultType());
        keyStore.load(null, "password".toCharArray());
        keyStore.setKeyEntry("client", clientKey, "".toCharArray(), new Certificate[]{clientCertificateChain});
        KeyManagerFactory keyManagerFactory = KeyManagerFactory.getInstance(KeyManagerFactory.getDefaultAlgorithm());
        keyManagerFactory.init(keyStore, "".toCharArray());

        KeyStore trustStore = KeyStore.getInstance(KeyStore.getDefaultType());
        trustStore.load(null, "password".toCharArray());
        trustStore.setCertificateEntry("ca", caCertificate);
        TrustManagerFactory trustManagerFactory = TrustManagerFactory.getInstance(TrustManagerFactory.getDefaultAlgorithm());
        trustManagerFactory.init(trustStore);

        SSLContext sslContext = SSLContext.getInstance("TLS");
        sslContext.init(keyManagerFactory.getKeyManagers(), trustManagerFactory.getTrustManagers(), null);

        HttpClient httpClient = HttpClient.newBuilder()
                .sslContext(sslContext)
                .build();

        HttpRequest request = HttpRequest.newBuilder()
                .GET()
                .uri(URI.create("https://prod.idrix.eu/secure/"))
                .build();

        HttpResponse<String> response = httpClient.send(request, HttpResponse.BodyHandlers.ofString());
        System.out.println(response.body());
    }

    public static String getContent(InputStream inputStream) throws IOException {
        try (InputStreamReader inputStreamReader = new InputStreamReader(Objects.requireNonNull(inputStream), StandardCharsets.UTF_8);
             BufferedReader bufferedReader = new BufferedReader(inputStreamReader)) {

            return bufferedReader.lines()
                    .collect(Collectors.joining(System.lineSeparator()));
        }
    }

}

I am using Java 11 and Bouncy Castle, see below for the specific artifact-id:

<dependency>
    <groupId>org.bouncycastle</groupId>
    <artifactId>bcpkix-jdk15on</artifactId>
    <version>1.68</version>
</dependency>

So for me it is not clear why the curl command works and the setup in java fails. Did I configured something wrong, any help would be great. I tried to provide as much as details possible. The private key is a test key so it won't harm to share it here.

Answer 1

I discovered what the issue was and so I wanted to share with others who have this in common.

It looks like on mac os x curl is by default using the system trusted certificates and on java side this is not the case. I am not sure what the behaviour of curl is with other operating systems. https://prod.idrix.eu/secure/ has a certificate which is signed by DST Root CA X3 I disabled this and rerun the exact same setup as described on my initial question and the curl https request fails, just like it also failed on java side.

I disabled it from keychain as shown here:

Https request with mutual authentication passes with curl but fails with java

Question

1 answers

solution1
1 ACCPTED 2021-01-23 10:16:02

Https request with mutual authentication passes with curl but fails with java

Question

1 answers

solution1 1 ACCPTED 2021-01-23 10:16:02

solution1
1 ACCPTED 2021-01-23 10:16:02