SSL handshake failure Server [FIN, ACK] after Client Hello

Question

Our company currently have an application created in VB6 running on a Windows 7 desktop computer. This application needs to connect to a web service using HTTPS.

For some reason this application can not connect to the target server. The target server is behind an Apache web server that also acts as a load balancer.

I traced the connection via wire shark and it seems that after the Client Hello the server responded with [FIN, ACK] than a Server Hello . Can anyone tell me what the problem is.

Client Hello & Server [FIN, ACK] [![Client HEllo][1]][1] [1]: https://i.stack.imgur.com/btXyX.jpg [![enter image description here][2]][2] [2]: https://i.stack.imgur.com/O5WDq.jpg

However, we can connect successfully in the following scenarios

1. If we use the same application under Windows 10. Please note when using Windows 10 our SSL version is TLS1.2.
2. If the application connect directly to the Application Server instead of the Apache Web Server. In this case our SSL protocol is still TLS1.0

I am guessing that we need to configure the Web Server to allow our application to connect. Our Administrator told us they already activated the TLS1.0 protocols so I am not entirely sure what is wrong now.

Why the server closed the connection, or any ideas what part of the logs or Web server configuration should I check to find out?

Please note that we are aware that we are using TLS1 protocol for this connection but this application is only for internal use and its replacement is already in development. However, there is a business need to make this work for the current environment.

Answer 1

In the case the Server doesn't support any of the Cipher-Suites suggested in the "Client Hello" packet, it may just close the connection.

You can check what is the returned Cipher-Suite in the case it works (when you use Windows10) and then investigate how to enable this Cipher-Suite in Windows7