TLSv1.2 PC => Android 5.0.2 handshake_failure

Question

Another TLS problem occured when connecting PC to Android Server (5.0.2) on TLSv1.2.

Everything is fine when it's the Android app that's trying to connect to the PC.

JCE is installed on the PC.

Below is the debug:

adding as trusted cert:
  Subject: CN=Root
  Issuer:  CN=Root

adding as trusted cert:
  Subject: CN=Signer
  Issuer:  CN=Root

trigger seeding of SecureRandom
done seeding SecureRandom
Thread-4, handling exception: java.net.SocketTimeoutException: Read timed out
Allow unsafe renegotiation: false
Allow legacy hello messages: true
Is initial handshake: true
Is secure renegotiation: false
Ignoring unsupported cipher suite: TLS_ECDHE_ECDSA_WITH_AES_256_CBC_SHA384 for TLSv1
Ignoring unsupported cipher suite: TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384 for TLSv1
Ignoring unsupported cipher suite: TLS_RSA_WITH_AES_256_CBC_SHA256 for TLSv1
Ignoring unsupported cipher suite: TLS_ECDH_ECDSA_WITH_AES_256_CBC_SHA384 for TLSv1
Ignoring unsupported cipher suite: TLS_ECDH_RSA_WITH_AES_256_CBC_SHA384 for TLSv1
Ignoring unsupported cipher suite: TLS_DHE_RSA_WITH_AES_256_CBC_SHA256 for TLSv1
Ignoring unsupported cipher suite: TLS_DHE_DSS_WITH_AES_256_CBC_SHA256 for TLSv1
Ignoring unsupported cipher suite: TLS_ECDHE_ECDSA_WITH_AES_256_CBC_SHA384 for TLSv1.1
Ignoring unsupported cipher suite: TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384 for TLSv1.1
Ignoring unsupported cipher suite: TLS_RSA_WITH_AES_256_CBC_SHA256 for TLSv1.1
Ignoring unsupported cipher suite: TLS_ECDH_ECDSA_WITH_AES_256_CBC_SHA384 for TLSv1.1
Ignoring unsupported cipher suite: TLS_ECDH_RSA_WITH_AES_256_CBC_SHA384 for TLSv1.1
Ignoring unsupported cipher suite: TLS_DHE_RSA_WITH_AES_256_CBC_SHA256 for TLSv1.1
Ignoring unsupported cipher suite: TLS_DHE_DSS_WITH_AES_256_CBC_SHA256 for TLSv1.1
%% No cached client session
*** ClientHello, TLSv1.2
RandomCookie:  GMT: 1421962449 bytes = { 141, 65, 196, 84, 116, 42, 129, 173, 178, 39, 249, 209, 192, 237, 117, 169, 182, 56, 220, 4, 35, 152, 182, 33, 238, 112, 79, 64 }
Session ID:  {}
Cipher Suites: [TLS_ECDHE_ECDSA_WITH_AES_256_CBC_SHA384, TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384, TLS_RSA_WITH_AES_256_CBC_SHA256, TLS_ECDH_ECDSA_WITH_AES_256_CBC_SHA384, TLS_ECDH_RSA_WITH_AES_256_CBC_SHA384, TLS_DHE_RSA_WITH_AES_256_CBC_SHA256, TLS_DHE_DSS_WITH_AES_256_CBC_SHA256, TLS_ECDHE_ECDSA_WITH_AES_256_CBC_SHA, TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA, TLS_RSA_WITH_AES_256_CBC_SHA, TLS_ECDH_ECDSA_WITH_AES_256_CBC_SHA, TLS_ECDH_RSA_WITH_AES_256_CBC_SHA, TLS_DHE_RSA_WITH_AES_256_CBC_SHA, TLS_DHE_DSS_WITH_AES_256_CBC_SHA, TLS_ECDHE_ECDSA_WITH_AES_128_CBC_SHA256, TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA256, TLS_RSA_WITH_AES_128_CBC_SHA256, TLS_ECDH_ECDSA_WITH_AES_128_CBC_SHA256, TLS_ECDH_RSA_WITH_AES_128_CBC_SHA256, TLS_DHE_RSA_WITH_AES_128_CBC_SHA256, TLS_DHE_DSS_WITH_AES_128_CBC_SHA256, TLS_ECDHE_ECDSA_WITH_AES_128_CBC_SHA, TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA, TLS_RSA_WITH_AES_128_CBC_SHA, TLS_ECDH_ECDSA_WITH_AES_128_CBC_SHA, TLS_ECDH_RSA_WITH_AES_128_CBC_SHA, TLS_DHE_RSA_WITH_AES_128_CBC_SHA, TLS_DHE_DSS_WITH_AES_128_CBC_SHA, TLS_ECDHE_ECDSA_WITH_AES_256_GCM_SHA384, TLS_ECDHE_ECDSA_WITH_AES_128_GCM_SHA256, TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384, TLS_RSA_WITH_AES_256_GCM_SHA384, TLS_ECDH_ECDSA_WITH_AES_256_GCM_SHA384, TLS_ECDH_RSA_WITH_AES_256_GCM_SHA384, TLS_DHE_RSA_WITH_AES_256_GCM_SHA384, TLS_DHE_DSS_WITH_AES_256_GCM_SHA384, TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256, TLS_RSA_WITH_AES_128_GCM_SHA256, TLS_ECDH_ECDSA_WITH_AES_128_GCM_SHA256, TLS_ECDH_RSA_WITH_AES_128_GCM_SHA256, TLS_DHE_RSA_WITH_AES_128_GCM_SHA256, TLS_DHE_DSS_WITH_AES_128_GCM_SHA256, TLS_ECDHE_ECDSA_WITH_3DES_EDE_CBC_SHA, TLS_ECDHE_RSA_WITH_3DES_EDE_CBC_SHA, SSL_RSA_WITH_3DES_EDE_CBC_SHA, TLS_ECDH_ECDSA_WITH_3DES_EDE_CBC_SHA, TLS_ECDH_RSA_WITH_3DES_EDE_CBC_SHA, SSL_DHE_RSA_WITH_3DES_EDE_CBC_SHA, SSL_DHE_DSS_WITH_3DES_EDE_CBC_SHA, TLS_EMPTY_RENEGOTIATION_INFO_SCSV]
Compression Methods:  { 0 }
Extension elliptic_curves, curve names: {secp256r1, sect163k1, sect163r2, secp192r1, secp224r1, sect233k1, sect233r1, sect283k1, sect283r1, secp384r1, sect409k1, sect409r1, secp521r1, sect571k1, sect571r1, secp160k1, secp160r1, secp160r2, sect163r1, secp192k1, sect193r1, sect193r2, secp224k1, sect239k1, secp256k1}
Extension ec_point_formats, formats: [uncompressed]
Extension signature_algorithms, signature_algorithms: SHA512withECDSA, SHA512withRSA, SHA384withECDSA, SHA384withRSA, SHA256withECDSA, SHA256withRSA, SHA224withECDSA, SHA224withRSA, SHA1withECDSA, SHA1withRSA, SHA1withDSA, MD5withRSA
***
Thread-30, WRITE: TLSv1.2 Handshake, length = 237
Thread-30, READ: TLSv1.2 Alert, length = 2
Thread-30, RECV TLSv1.2 ALERT:  fatal, handshake_failure
Thread-30, called closeSocket()
Thread-30, handling exception: javax.net.ssl.SSLHandshakeException: Received fatal alert: handshake_failure
javax.net.ssl.SSLHandshakeException: Received fatal alert: handshake_failure
    sun.security.ssl.Alerts.getSSLException(Alerts.java:192)
    sun.security.ssl.Alerts.getSSLException(Alerts.java:154)
    sun.security.ssl.SSLSocketImpl.recvAlert(SSLSocketImpl.java:2023)
    sun.security.ssl.SSLSocketImpl.readRecord(SSLSocketImpl.java:1125)
    sun.security.ssl.SSLSocketImpl.performInitialHandshake(SSLSocketImpl.java:1375)
    sun.security.ssl.SSLSocketImpl.startHandshake(SSLSocketImpl.java:1403)
    sun.security.ssl.SSLSocketImpl.startHandshake(SSLSocketImpl.java:1387)

How to resolve this issue?

Server

    System.setProperty("https.protocols", "TLSv1.2");
    System.setProperty("jdk.tls.client.protocols", "TLSv1.2");

    KeyManagerFactory keyManagerFactory;
    keyManagerFactory = KeyManagerFactory.getInstance("PKIX");
    keyManagerFactory.init(keyStore, keyPairPassword);

    TrustManagerFactory trustManagerFactory = TrustManagerFactory.getInstance("PKIX");
    trustManagerFactory.init(trustStore);

    SSLContext sslContext = SSLContext.getInstance("TLSv1.2");
    sslContext.init(keyManagerFactory.getKeyManagers(), trustManagerFactory.getTrustManagers(), null);

    SSLServerSocketFactory sslServerSocketFactory = sslContext.getServerSocketFactory();

    SSLServerSocket sslServerSocket
            = (SSLServerSocket) sslServerSocketFactory.createServerSocket(port);
    sslServerSocket.setEnabledCipherSuites(new String[]{"TLS_DHE_RSA_WITH_AES_128_CBC_SHA256",
    "TLS_DHE_RSA_WITH_AES_128_GCM_SHA256",
    "TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA256",
    "TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA256"});
    sslServerSocket.setNeedClientAuth(true);

Client

    System.setProperty("https.protocols", "TLSv1.2");
    System.setProperty("jdk.tls.client.protocols", "TLSv1.2");
    KeyStore keyStore = getKeyStore();

    KeyManagerFactory keyManagerFactory;
    keyManagerFactory = KeyManagerFactory.getInstance("PKIX");
    keyManagerFactory.init(keyStore, getPassword());

    KeyStore trustStore = getTrustStore();
    TrustManagerFactory trustManagerFactory;
    trustManagerFactory = TrustManagerFactory.getInstance("PKIX");
    trustManagerFactory.init(trustStore);

    SSLContext sslContext = SSLContext.getInstance("TLSv1.2");
    sslContext.init(keyManagerFactory.getKeyManagers(), trustManagerFactory.getTrustManagers(), null);

    SSLSocket socket = (SSLSocket) sslContext.getSocketFactory().createSocket(
    ipAddress, port);
    socket.setTcpNoDelay(true);
    socket.setKeepAlive(false);
    socket.setNeedClientAuth(true);
    socket.startHandshake();

Answer 1

As indicated by https://blogs.oracle.com/java-platform-group/entry/diagnosing_tls_ssl_and_https this error happens when the client and server have no algorithms in common.

Note that this is a bit misleading: client must not only have an algorithm in common but also an elliptic curve (if using elliptic curves).

The ClientHello list the supported curves in

Extension elliptic_curves, curve names: ...

However if the counterpart (server) requires a curve which is not supported by you (the client), for example brainpoolP256r1 you'll get exactly this error.

To check on that, install wireshark capture the different ClientHello and check for differences. Furthermore you can check the server for the supported curves by using openSSL. For example in case of brainpoolP256r1 which is supported by openSSL (but not Java, even JDK 1.8 with JCE Unlimited Strength) you can do

openssl s_client -connect YOUR_URL[:PORT] -tls1_2

In case of a successful connection you will an output including something like

No client certificate CA names sent
Client Certificate Types: RSA sign, DSA sign, ECDSA sign
Requested Signature Algorithms: RSA+SHA512:DSA+SHA512:ECDSA...
Shared Requested Signature Algorithms: RSA+SHA512:DSA+SHA51...
Peer signing digest: SHA512
Server Temp Key: ECDH, brainpoolP256r1, 256 bits
---
SSL handshake has read 2511 bytes and written 455 bytes
---
New, TLSv1/SSLv3, Cipher is ECDHE-ECDSA-AES256-GCM-SHA384

where the curve is indicated by

Server Temp Key: ...

BouncyCastle supports numerous algorithms not supported by standard Java. Try to use BouncyCastle for establishing TLS connection.