SSL handshake_failure on Websphere 8.5 (working on Tomcat)

Question

Long story short; is there any reason why an application working on a Tomcat isn't able to communicate with Paypal servers? Some background: we are developing a series of portlets on Liferay, which at some point communicate with paypal servers to start and validate a purchasing process. This works like charm on local tomcat without any special configuration, but after installing Liferay and the portlet it fails to start the process. The stack looks like this:

[16/12/14 13:51:01:728 GMT+01:00] 0000015d SystemOut     O 13:51:01,727 ERROR [WebContainer : 2][render_portlet_jsp:132] null
javax.net.ssl.SSLHandshakeException: Received fatal alert: handshake_failure
    at com.ibm.jsse2.o.a(o.java:33)
    at com.ibm.jsse2.o.a(o.java:30)
    at com.ibm.jsse2.SSLSocketImpl.b(SSLSocketImpl.java:168)
    at com.ibm.jsse2.SSLSocketImpl.a(SSLSocketImpl.java:318)
    at com.ibm.jsse2.SSLSocketImpl.h(SSLSocketImpl.java:403)
    at com.ibm.jsse2.SSLSocketImpl.a(SSLSocketImpl.java:431)
    at com.ibm.jsse2.SSLSocketImpl.startHandshake(SSLSocketImpl.java:315)
    at com.ibm.net.ssl.www2.protocol.https.c.afterConnect(c.java:103)
    at com.ibm.net.ssl.www2.protocol.https.d.connect(d.java:42)
    at sun.net.www.protocol.http.HttpURLConnection.getInputStream(HttpURLConnection.java:1184)
    at java.net.HttpURLConnection.getResponseCode(HttpURLConnection.java:390)
    at com.ibm.net.ssl.www2.protocol.https.b.getResponseCode(b.java:82)
    at com.paypal.core.HttpConnection.execute(HttpConnection.java:93)
    at com.paypal.core.APIService.makeRequestUsing(APIService.java:176)
    at com.paypal.core.BaseService.call(BaseService.java:265)
    at urn.ebay.api.PayPalAPI.PayPalAPIInterfaceServiceService.setExpressCheckout(PayPalAPIInterfaceServiceService.java:2196)
    at urn.ebay.api.PayPalAPI.PayPalAPIInterfaceServiceService.setExpressCheckout(PayPalAPIInterfaceServiceService.java:2148)

I've been "googleing" a bit around and can't figure out where the error can be. We have tried to register signer the Verisign certificate retrieved from paypal site, but nothing changed.

May somebody aim us on the right direction? Thanks!

---

UPDATE After rising the network logging level I can see the following in the logs:

 O class com.ibm.websphere.ssl.protocol.SSLSocketFactory is loaded O instantiated an instance of class com.ibm.websphere.ssl.protocol.SSLSocketFactory O handshake: true O Ignoring unsupported cipher suite: SSL_RSA_WITH_AES_128_CBC_SHA O Ignoring unsupported cipher suite: SSL_DHE_RSA_WITH_AES_128_CBC_SHA O Ignoring unsupported cipher suite: SSL_DHE_DSS_WITH_AES_128_CBC_SHA O %% No cached client session O *** ClientHello, SSLv3 O RandomCookie: GMT: 1402031796 bytes = { 166, 100, 171, 183, 214, 31, 12, 68, 124, 68, 151, 195, 7, 4, 28, 112, 39, 90, 248, 143, 129, 106, 212, 33, 244, 40, 233, 94 } O Session ID: {} O Cipher Suites: [SSL_RSA_WITH_RC4_128_MD5, SSL_RSA_WITH_RC4_128_SHA, SSL_RSA_WITH_3DES_EDE_CBC_SHA, SSL_RSA_FIPS_WITH_3DES_EDE_CBC_SHA, SSL_DHE_RSA_WITH_3DES_EDE_CBC_SHA, SSL_DHE_DSS_WITH_3DES_EDE_CBC_SHA, SSL_DHE_DSS_WITH_RC4_128_SHA, SSL_RSA_WITH_DES_CBC_SHA, SSL_RSA_FIPS_WITH_DES_CBC_SHA, SSL_DHE_RSA_WITH_DES_CBC_SHA, SSL_DHE_DSS_WITH_DES_CBC_SHA, SSL_RSA_EXPORT_WITH_RC4_40_MD5, SSL_RSA_EXPORT_WITH_DES40_CBC_SHA, SSL_DHE_RSA_EXPORT_WITH_DES40_CBC_SHA, SSL_DHE_DSS_EXPORT_WITH_DES40_CBC_SHA, SSL_RENEGO_PROTECTION_REQUEST] O Compression Methods: { 0 } O *** O [write] MD5 and SHA1 hashes: len = 75 O 0000: 01 00 00 47 03 00 54 91 4f b4 a6 64 ab b7 d6 1f ...G..TO.d.... 0010: 0c 44 7c 44 97 c3 07 04 1c 70 27 5a f8 8f 81 6a .DD....pZ..j 0020: d4 21 f4 28 e9 5e 00 00 20 00 04 00 05 00 0a fe ................ 0030: ff 00 16 00 13 00 66 00 09 fe fe 00 15 00 12 00 ......f......... 0040: 03 00 08 00 14 00 11 00 ff 01 00 ........... O WebContainer : 10, WRITE: SSLv3 Handshake, length = 75 O [Raw write]: length = 80 O 0000: 16 03 00 00 4b 01 00 00 47 03 00 54 91 4f b4 a6 ....K...G..TO. 0010: 64 ab b7 d6 1f 0c 44 7c 44 97 c3 07 04 1c 70 27 d.....DD....p. 0020: 5a f8 8f 81 6a d4 21 f4 28 e9 5e 00 00 20 00 04 Z...j........... 0030: 00 05 00 0a fe ff 00 16 00 13 00 66 00 09 fe fe ...........f.... 0040: 00 15 00 12 00 03 00 08 00 14 00 11 00 ff 01 00 ................ O [Raw read]: length = 5 O 0000: 15 03 00 00 02 ..... O [Raw read]: length = 2 O 0000: 02 28 .. O WebContainer : 10, READ: SSLv3 Alert, length = 2 O WebContainer : 10, RECV TLSv1 ALERT: fatal, handshake_failure O WebContainer : 10, called closeSocket() O WebContainer : 10, handling exception: javax.net.ssl.SSLHandshakeException: Received fatal alert: handshake_failure O handshake: true O Ignoring unsupported cipher suite: SSL_RSA_WITH_AES_128_CBC_SHA O Ignoring unsupported cipher suite: SSL_DHE_RSA_WITH_AES_128_CBC_SHA O Ignoring unsupported cipher suite: SSL_DHE_DSS_WITH_AES_128_CBC_SHA O %% No cached client session O *** ClientHello, SSLv3 O RandomCookie: GMT: 1402031797 bytes = { 153, 95, 153, 155, 68, 36, 152, 92, 71, 172, 226, 104, 156, 107, 235, 73, 63, 239, 198, 202, 166, 216, 158, 26, 45, 59, 169, 169 } O Session ID: {} O Cipher Suites: [SSL_RSA_WITH_RC4_128_MD5, SSL_RSA_WITH_RC4_128_SHA, SSL_RSA_WITH_3DES_EDE_CBC_SHA, SSL_RSA_FIPS_WITH_3DES_EDE_CBC_SHA, SSL_DHE_RSA_WITH_3DES_EDE_CBC_SHA, SSL_DHE_DSS_WITH_3DES_EDE_CBC_SHA, SSL_DHE_DSS_WITH_RC4_128_SHA, SSL_RSA_WITH_DES_CBC_SHA, SSL_RSA_FIPS_WITH_DES_CBC_SHA, SSL_DHE_RSA_WITH_DES_CBC_SHA, SSL_DHE_DSS_WITH_DES_CBC_SHA, SSL_RSA_EXPORT_WITH_RC4_40_MD5, SSL_RSA_EXPORT_WITH_DES40_CBC_SHA, SSL_DHE_RSA_EXPORT_WITH_DES40_CBC_SHA, SSL_DHE_DSS_EXPORT_WITH_DES40_CBC_SHA, SSL_RENEGO_PROTECTION_REQUEST] O Compression Methods: { 0 } O *** O [write] MD5 and SHA1 hashes: len = 75 O 0000: 01 00 00 47 03 00 54 91 4f b5 99 5f 99 9b 44 24 ...G..TO....D. 0010: 98 5c 47 ac e2 68 9c 6b eb 49 3f ef c6 ca a6 d8 ..G..hkI..... 0020: 9e 1a 2d 3b a9 a9 00 00 20 00 04 00 05 00 0a fe ................ 0030: ff 00 16 00 13 00 66 00 09 fe fe 00 15 00 12 00 ......f......... 0040: 03 00 08 00 14 00 11 00 ff 01 00 ........... O WebContainer : 10, WRITE: SSLv3 Handshake, length = 75 O [Raw write]: length = 80 O 0000: 16 03 00 00 4b 01 00 00 47 03 00 54 91 4f b5 99 ....K...G..TO. 0010: 5f 99 9b 44 24 98 5c 47 ac e2 68 9c 6b eb 49 3f ...D...G..hkI 0020: ef c6 ca a6 d8 9e 1a 2d 3b a9 a9 00 00 20 00 04 ................ 0030: 00 05 00 0a fe ff 00 16 00 13 00 66 00 09 fe fe ...........f.... 0040: 00 15 00 12 00 03 00 08 00 14 00 11 00 ff 01 00 ................ O [Raw read]: length = 5 O 0000: 15 03 00 00 02 ..... O [Raw read]: length = 2 O 0000: 02 28 .. O WebContainer : 10, READ: SSLv3 Alert, length = 2 O WebContainer : 10, RECV TLSv1 ALERT: fatal, handshake_failure O WebContainer : 10, called closeSocket() O WebContainer : 10, handling exception: javax.net.ssl.SSLHandshakeException: Received fatal alert: handshake_failure O 10:41:09,593 ERROR [WebContainer : 10][PaypalUtils:145] Errores en setPaypalExpressCheckout javax.net.ssl.SSLHandshakeException: Received fatal alert: handshake_failure at com.ibm.jsse2.oa(o.java:33) at com.ibm.jsse2.oa(o.java:30)

The SSL protocol configured in the server console is TLS.

Answer 1

Received fatal alert: handshake_failure

This can be anything, like missing client certificate, no shared ciphers, wrong protocol version etc. But it has usually nothing to do with certificate validation.

Please check with other clients, check with SSLLabs etc to reduce the number of possible reasons. See alsohttp://noxxi.de/howto/ssl-debugging.html#aid_external_debugging for steps you might try in debugging and which information you should collect if you need help from others. If you have more information post it here so one could hopefully find a solution for your problem.

EDIT based on new information in the question:

 O *** ClientHello, SSLv3
 ...
 O 0000: 16 03 00

You are using SSL 3.0 which can be seen from the debug messages. SSL 3.0 is blocked by most major sites because of POODLE. While you claim to use TLS1.x already this seems not to be the case according to these debugging information, so you should check your setup again.

Answer 2

Usually in Websphere you need to import the certificate of the server you want to communicate with. If you try it by hand probably you will not import the appropiate cert in the chain so rather use "retrieve from port" in the admin console.

Another option is that Paypal is killing SSL due to the Poodle issue :). Raise your security level to TLS in that case in the security menu in the admin console

Answer 3

The trace shows that you received a handshake failure from the remote server. I can see that your server sent a client hello message, so you would expect to see a response with a server hello message, instead the connection ended abruptly from the other end. In these cases you would need someone from the remote side to take a look and tell you why this connection is failing. Otherwise, you're really working in the dark and you can try many different things.

However, one thing I'm noticing is that you're using SSLv3 protocol while the other end seems to be using TLSv1 protocol. I would assume an entity like Paypal is probably FIPS compliant which means they stopped using SSL protocol and only use TLS as part of being compliant.