Java-到IIS 7.5服务器的相互身份验证-连接重置-对等方未经过身份验证

Question

I am building a java client to connect to a Windows IIS 7.5 Server. Right now I have control over both systems, as I am building a proof of concept so the windows machine is a VM on my laptop. I have successfully setup the Client Certificates in IIS 7.5 and verified that Mutual Authentication via self-signed certificates are working via the browser.

Unfortunately, after the SSLv2 client hello message and TLSv1 client hello message are sent, the connection is being reset with a TLSv1 Alert of unexpected_message. I have searched StackOverflow as well as many other places to fix this issue and cannot seem to find a solution to this particular problem. This is the first time I have use Mutual Auth though. My code is as follows:

final KeyStore keystore = KeyStore.getInstance("pkcs12");
        InputStream keystoreInput = new FileInputStream(KEY_STORE_PATH);
        keystore.load(keystoreInput, "test".toCharArray());
        System.out.println("Keystore has " + keystore.size() + " keys");
        // load the trustore, leave it null to rely on cacerts distributed with the JVM - truststore type is probably jks or pkcs12
        KeyStore truststore = KeyStore.getInstance("pkcs12");
        InputStream truststoreInput = new FileInputStream(TRUST_STORE_PATH);
        truststore.load(truststoreInput, "test".toCharArray());
        System.out.println("Truststore has " + truststore.size() + " keys");


        URI uri = new URI("https://192.168.2.112:444");
        String result = null; 
        DefaultHttpClient httpclient = new DefaultHttpClient();

                SSLSocketFactory socketFactory = new SSLSocketFactory(keystore, String.valueOf("test"), truststore); 
                Scheme sch = new Scheme("https", 444, socketFactory); 

                httpclient.getConnectionManager().getSchemeRegistry().register(sch); 

                HttpGet httpget = new HttpGet(uri.toASCIIString()); 

                HttpResponse response = httpclient.execute(httpget); 
                HttpEntity entity = response.getEntity(); 

And the debug trace is here:

    Keystore has 1 keys
Truststore has 1 keys
***
found key for : pvktmp:0bfeeb5c-bf33-4ef3-a00e-d547e35774c6
chain [0] = [
[
  Version: V3
  Subject: CN=Chris Nanda
  Signature Algorithm: SHA1withRSA, OID = 1.3.14.3.2.29

  Key:  Sun RSA public key, 2048 bits
  modulus: 27130899851248154939882863851763653399730366371287600333384386175121736158788923280330198649817012144950949568512692167276184169703299725013344337362317100916091834207526372858824337248769441540975936119155896345867494063216196776658828993122572447899402194194077727951304168585241790036601092365261322113697370538176270611427949951453920476169582825252258759745485892192237387988896185894058602081306722996452816186803769762977498453060320624472197936928709207159769282155656635385394975574974944302095509596248869870407415024524635233253519215244773231069920494280237259291072815104686895907061226740651660601534821
  public exponent: 65537
  Validity: [From: Sat Jan 21 00:00:00 CST 2012,
               To: Tue Jan 21 00:00:00 CST 2014]
  Issuer: CN=BHS CA
  SerialNumber: [   -6e5243e9 73783768 b67202d3 8e7bd30e]

Certificate Extensions: 2
[1]: ObjectId: 2.5.29.1 Criticality=false
Extension unknown: DER encoded OCTET string =
0000: 04 3B 30 39 80 10 B2 DC   54 DD B5 45 40 62 7C B7  .;09....T..E@b..
0010: F1 73 76 53 54 14 A1 13   30 11 31 0F 30 0D 06 03  .svST...0.1.0...
0020: 55 04 03 13 06 42 48 53   20 43 41 82 10 07 BE 71  U....BHS CA....q
0030: 73 48 F3 F1 85 48 20 1C   EE 05 07 EF 20           sH...H ..... 


[2]: ObjectId: 2.5.29.37 Criticality=false
ExtendedKeyUsages [
  clientAuth
]

]
  Algorithm: [SHA1withRSA]
  Signature:
0000: 43 D0 3A 1C 15 DC A2 67   5A 14 44 10 78 CF 30 7C  C.:....gZ.D.x.0.
0010: AB FB 95 65 E5 5E 9F D1   40 BE 1A 5E 05 D5 CB 59  ...e.^..@..^...Y
0020: C9 D3 08 F8 DD E7 65 05   C5 22 9C A9 23 E1 4A A1  ......e.."..#.J.
0030: 71 B1 BF 09 70 1D 64 FA   94 92 51 0B 44 A2 48 4C  q...p.d...Q.D.HL
0040: 3B 4A 9B F2 E1 8C 21 D9   0E 45 BB 0E 28 9B 67 9C  ;J....!..E..(.g.
0050: A0 FC 5E DD 79 77 74 16   CB 81 F7 36 C5 33 EE 14  ..^.ywt....6.3..
0060: 85 CA F4 4E 0D 2D AF 59   C0 4E 49 58 79 8C 83 9F  ...N.-.Y.NIXy...
0070: 77 F7 DB 26 53 F0 FF 1A   F5 CB DD 44 05 FC 3B 9E  w..&S......D..;.
0080: D2 19 AF 95 B1 BA 9E 78   4A 72 84 77 32 17 68 88  .......xJr.w2.h.
0090: CC B4 04 FB C4 3F 4C D5   F4 5C B5 23 9F 78 2C A9  .....?L..\.#.x,.
00A0: A2 EB 4D 3E 95 09 B9 A8   38 35 51 95 3C B9 48 5F  ..M>....85Q.<.H_
00B0: D3 40 7B 5B 40 CE F5 47   09 30 E0 B8 AB B5 7A 61  .@.[@..G.0....za
00C0: 6D C0 11 36 5B 07 1B A4   96 B2 41 DF 6E C2 91 23  m..6[.....A.n..#
00D0: D3 D5 37 18 79 77 C6 F4   30 C4 29 C7 8E 25 63 E2  ..7.yw..0.)..%c.
00E0: 92 A3 53 C2 37 1E C4 8C   2C 8C FD 93 63 62 F4 1B  ..S.7...,...cb..
00F0: 01 36 B9 60 8F 70 DD A2   E1 25 20 78 7A AE 51 84  .6.`.p...% xz.Q.

]
***
adding as trusted cert:
  Subject: CN=BHS CA
  Issuer:  CN=BHS CA
  Algorithm: RSA; Serial number: 0x7be717348f3f18548201cee0507ef20
  Valid from Sat Jan 21 00:00:00 CST 2012 until Thu Jan 21 00:00:00 CST 2016

trigger seeding of SecureRandom
done seeding SecureRandom
trustStore is: /System/Library/Java/JavaVirtualMachines/1.6.0.jdk/Contents/Home/lib/security/cacerts
trustStore type is : jks
trustStore provider is : 
init truststore
adding as trusted cert:
  Subject: CN=Secure Global CA, O=SecureTrust Corporation, C=US
  Issuer:  CN=Secure Global CA, O=SecureTrust Corporation, C=US
  Algorithm: RSA; Serial number: 0x75622a4e8d48a894df413c8f0f8eaa5
  Valid from Tue Nov 07 13:42:28 CST 2006 until Mon Dec 31 13:52:06 CST 2029



adding as trusted cert:
  Subject: CN=TC TrustCenter Class 2 CA II, OU=TC TrustCenter Class 2 CA, O=TC TrustCenter GmbH, C=DE
  Issuer:  CN=TC TrustCenter Class 2 CA II, OU=TC TrustCenter Class 2 CA, O=TC TrustCenter GmbH, C=DE
  Algorithm: RSA; Serial number: 0x2e6a000100021fd752212c115c3b
  Valid from Thu Jan 12 08:38:43 CST 2006 until Wed Dec 31 16:59:59 CST 2025

adding as trusted cert:
  Subject: CN=VRK Gov. Root CA, OU=Varmennepalvelut, OU=Certification Authority Services, O=Vaestorekisterikeskus CA, ST=Finland, C=FI
  Issuer:  CN=VRK Gov. Root CA, OU=Varmennepalvelut, OU=Certification Authority Services, O=Vaestorekisterikeskus CA, ST=Finland, C=FI
  Algorithm: RSA; Serial number: 0x186a0
  Valid from Wed Dec 18 07:53:00 CST 2002 until Mon Dec 18 07:51:08 CST 2023

trigger seeding of SecureRandom
done seeding SecureRandom
main, setSoTimeout(0) called
Allow unsafe renegotiation: false
Allow legacy hello messages: true
Is initial handshake: true
Is secure renegotiation: false
%% No cached client session
*** ClientHello, TLSv1
RandomCookie:  GMT: 1339396239 bytes = { 174, 62, 41, 222, 119, 84, 174, 118, 101, 33, 7, 220, 249, 96, 130, 234, 41, 163, 171, 6, 194, 145, 45, 132, 134, 39, 186, 32 }
Session ID:  {}
Cipher Suites: [SSL_RSA_WITH_RC4_128_MD5, SSL_RSA_WITH_RC4_128_SHA, TLS_RSA_WITH_AES_128_CBC_SHA, TLS_RSA_WITH_AES_256_CBC_SHA, TLS_DHE_RSA_WITH_AES_128_CBC_SHA, TLS_DHE_RSA_WITH_AES_256_CBC_SHA, TLS_DHE_DSS_WITH_AES_128_CBC_SHA, TLS_DHE_DSS_WITH_AES_256_CBC_SHA, SSL_RSA_WITH_3DES_EDE_CBC_SHA, SSL_DHE_RSA_WITH_3DES_EDE_CBC_SHA, SSL_DHE_DSS_WITH_3DES_EDE_CBC_SHA, SSL_RSA_WITH_DES_CBC_SHA, SSL_DHE_RSA_WITH_DES_CBC_SHA, SSL_DHE_DSS_WITH_DES_CBC_SHA, SSL_RSA_EXPORT_WITH_RC4_40_MD5, SSL_RSA_EXPORT_WITH_DES40_CBC_SHA, SSL_DHE_RSA_EXPORT_WITH_DES40_CBC_SHA, SSL_DHE_DSS_EXPORT_WITH_DES40_CBC_SHA, TLS_EMPTY_RENEGOTIATION_INFO_SCSV]
Compression Methods:  { 0 }
***
[write] MD5 and SHA1 hashes:  len = 81
0000: 01 00 00 4D 03 01 50 D6   91 8F AE 3E 29 DE 77 54  ...M..P....>).wT
0010: AE 76 65 21 07 DC F9 60   82 EA 29 A3 AB 06 C2 91  .ve!...`..).....
0020: 2D 84 86 27 BA 20 00 00   26 00 04 00 05 00 2F 00  -..'. ..&...../.
0030: 35 00 33 00 39 00 32 00   38 00 0A 00 16 00 13 00  5.3.9.2.8.......
0040: 09 00 15 00 12 00 03 00   08 00 14 00 11 00 FF 01  ................
0050: 00                                                 .
main, WRITE: TLSv1 Handshake, length = 81
[write] MD5 and SHA1 hashes:  len = 110
0000: 01 03 01 00 45 00 00 00   20 00 00 04 01 00 80 00  ....E... .......
0010: 00 05 00 00 2F 00 00 35   00 00 33 00 00 39 00 00  ..../..5..3..9..
0020: 32 00 00 38 00 00 0A 07   00 C0 00 00 16 00 00 13  2..8............
0030: 00 00 09 06 00 40 00 00   15 00 00 12 00 00 03 02  .....@..........
0040: 00 80 00 00 08 00 00 14   00 00 11 00 00 FF 50 D6  ..............P.
0050: 91 8F AE 3E 29 DE 77 54   AE 76 65 21 07 DC F9 60  ...>).wT.ve!...`
0060: 82 EA 29 A3 AB 06 C2 91   2D 84 86 27 BA 20        ..).....-..'. 
main, WRITE: SSLv2 client hello message, length = 110
[Raw write]: length = 112
0000: 80 6E 01 03 01 00 45 00   00 00 20 00 00 04 01 00  .n....E... .....
0010: 80 00 00 05 00 00 2F 00   00 35 00 00 33 00 00 39  ....../..5..3..9
0020: 00 00 32 00 00 38 00 00   0A 07 00 C0 00 00 16 00  ..2..8..........
0030: 00 13 00 00 09 06 00 40   00 00 15 00 00 12 00 00  .......@........
0040: 03 02 00 80 00 00 08 00   00 14 00 00 11 00 00 FF  ................
0050: 50 D6 91 8F AE 3E 29 DE   77 54 AE 76 65 21 07 DC  P....>).wT.ve!..
0060: F9 60 82 EA 29 A3 AB 06   C2 91 2D 84 86 27 BA 20  .`..).....-..'. 
main, handling exception: java.net.SocketException: Connection reset
main, SEND TLSv1 ALERT:  fatal, description = unexpected_message
main, WRITE: TLSv1 Alert, length = 2
main, Exception sending alert: java.net.SocketException: Broken pipe
main, called closeSocket()
main, IOException in getSession():  java.net.SocketException: Connection reset
main, called close()
main, called closeInternal(true)
main, called close()
main, called closeInternal(true)
peer not authenticated

Same error every time, so I am assuming there is a detail either in IIS that I am missing, or a flag of some type in java that I am missing. As I said, this works in the browser with the loaded certificates.

What is causing this handshake error and is there something incorrect about the code above? There is no ServerHello being returned during the handshake.

Answer 1

I gave the exact same error when I try to consume IIS service from axis 2 client stub with X509 certificate based authentication.

After i changed the transport channel implementation from the default httpclient3 to the newer httpclient4, the issue is gone.

Ps: and try to force SSLContext to TLSv1.2... You can view my working test implementation here