"Defective token detected" error (NTLM not Kerberos) with Kerberos/Spring Security/IE/Active Directory

Question

We are having trouble getting Spring Security/Kerberos/AD to work for our web app. Our diagnosis is that our AD server sending an NTLM token (we can tell as it starts with "TlRMTVNT.....") to IE and IE is then sending this to our application and it's failing. Our AD server should be sending a Kerberos/SPNEGO token to IE.

The "moving parts" are as follows:

• Spring Security 3.0 (patched)
• Microsoft Windows Server Enterprise 2003 SP1 Active Directory
• IE 8
• Tomcat (TC Server 6.0)
• Java 1.6

We have set everything up as detailed in the instructions here:

https://spring.io/blog/2009/09/28/spring-security-kerberos-spnego-extension

This involved:

• Creating a normal user as the Service Principal (the same as the machine name where our application resides). We set the following account options:
  • disabled 'Use has to change password at next logon'
  • enabled 'password never expires'
  • enabled 'Use Kerberos DES…'
  • disabled 'Do not require Kerberos preauthentication'
  • NOTE: Server 2003 does not present the 'This account supports Kerberos AES 128 bit…' and 'This account supports Kerberos AES 256 bit…' options
• Used "ktpass.exe" to assign the service principal name (SPN) to this new user and export this user key to a keytab file. using 'ktpass /out ourweb.keytab /mapuser ourweb.testdomain.ourcompany.co.uk@TESTDOMAIN.OURCOMPANY.CO.UK /princ HTTP/ourweb.testdomain.ourcompany.co.uk@TESTDOMAIN.OURCOMPANY.CO.UK /pass *'
• Downloaded source code from https://src.springframework.org/svn/se-security/trunk .
• Copied the keytab file from the AD server to WEB-INF/etc of source code (application).
• Made change to file SunJaasKerbersoTicketValidator.java to read keytab file. (To resolve the bug in which the application can't read the keytab file from the Java classpath) options.put("keyTab", "C:\se-security\spring-security-kerberos\spring-security-kerberos-sample\src\main\webapp\WEB-INF\etc\ourweb.keytab");
• Configured web.xml to use spnego.xml. contextConfigLocation /WEB-INF/spnego.xml
  • Configured Spring Security (spnego.xml) to use Kerberos (SpnegoEntryPoint, SpnegoAuthenticationProcessingFilter and KerberosServiceAuthenticationProvider beans) by providing our service prinicipal name and keytab file location.
  • Configured spnego.xml to read keytab file copied in WEB-INF/etc.

When we started up our TC server we could see things initializing nicely (ie without error - "principles key obtained from the keytab"):

Creating instance of bean 'org.springframework.security.extensions.kerberos.SunJaasKerberosTicketValidator#10fa4b8' 
Invoking afterPropertiesSet() on bean with name 'org.springframework.security.extensions.kerberos.SunJaasKerberosTicketValidator#10fa4b8' 
Config name: C:\WINDOWS\krb5.ini
Debug is  true storeKey true useTicketCache false useKeyTab true doNotPrompt true ticketCache is null isInitiator false KeyTab is C:\se-security\spring-security-kerberos\spring-security-kerberos-sample\src\main\webapp\WEB-INF\etc\ourwebapp4.keytab refreshKrb5Config is false principal is HTTP/ourwebappweb4.testdomain.ourcompany.co.uk tryFirstPass is false useFirstPass is false storePass is false clearPass is false
>>> KeyTabInputStream, readName(): TESTDOMAIN.OURCOMPANY.CO.UK
>>> KeyTabInputStream, readName(): HTTP
>>> KeyTabInputStream, readName(): ourweb
>>> KeyTab: load() entry length: 78; type: 1
>>> KeyTabInputStream, readName(): TESTDOMAIN.OURCOMPANY.CO.UK
>>> KeyTabInputStream, readName(): HTTP
>>> KeyTabInputStream, readName(): ourweb.testdomain.ourcompany.co.uk
>>> KeyTab: load() entry length: 113; type: 1
Added key: 1version: 2
Ordering keys wrt default_tkt_enctypes list
default etypes for default_tkt_enctypes: 1.
0: EncryptionKey: keyType=1 kvno=2 keyValue (hex dump)=
0000: 91 01 43 E3 02 A8 B9 83   

principal's key obtained from the keytab
principal is HTTP/ourweb.testdomain.ourcompany.co.uk@TESTDOMAIN.OURCOMPANY.CO.UK
EncryptionKey: keyType=1 keyBytes (hex dump)=0000: 91 01 43 E3 02 A8 B9 83   
Added server's keyKerberos Principal HTTP/ourweb.testdomain.ourcompany.co.uk@TESTDOMAIN.OURCOMPANY.CO.UKKey Version 2key EncryptionKey: keyType=1 keyBytes (hex dump)=
0000: 91 01 43 E3 02 A8 B9 83   

[Krb5LoginModule] added Krb5Principal  HTTP/ourweb.testdomain.ourcompany.co.uk@TESTDOMAIN.OURCOMPANY.CO.UK to Subject Commit Succeeded 

Finished creating instance of bean 'org.springframework.security.extensions.kerberos.SunJaasKerberosTicketValidator#10fa4b8' 

Ready to test, we then enabled "Windows Integrated Authentication" in IE and made sure that the domain was listed in IE's local intr.net site section. We then connected to our web application using the fully qualified domain name.

When we did so we got the following errors in the browser:

500 Internal server error.

and in the TC Server log file:

Negotiate Header was invalid: Negotiate     TlRMTVNTUAABAAAAB4IIogAAAAAAAAAAAAAAAAAAAAAFASgKAAAADw== 
  org.springframework.security.authentication.BadCredentialsException: Kerberos validation not succesfull
  at org.springframework.security.extensions.kerberos.SunJaasKerberosTicketValidator.validateTicket(SunJaasKerberosTicketValidator.java:74)
  at org.springframework.security.extensions.kerberos.KerberosServiceAuthenticationProvider.authenticate(KerberosServiceAuthenticationProvider.java:92)
  at org.springframework.security.authentication.ProviderManager.doAuthentication(ProviderManager.java:120)
  at org.springframework.security.authentication.AbstractAuthenticationManager.authenticate(AbstractAuthenticationManager.java:48)
  at org.springframework.security.extensions.kerberos.web.SpnegoAuthenticationProcessingFilter.doFilter(SpnegoAuthenticationProcessingFilter.java:132)
  at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:355)
  at org.springframework.security.web.context.SecurityContextPersistenceFilter.doFilter(SecurityContextPersistenceFilter.java:79)
  at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:355)
  at org.springframework.security.web.FilterChainProxy.doFilter(FilterChainProxy.java:149)
  at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:237)
  at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:167)
  at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:235)
  at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206)
  at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:233)
  at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:191)
  at com.springsource.metrics.collection.web.HttpRequestMetricCollectionValve.invoke(HttpRequestMetricCollectionValve.java:44)
  at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:128)
  at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:102)
  at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:109)
  at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:293)
  at org.apache.coyote.http11.Http11Processor.process(Http11Processor.java:849)
  at org.apache.coyote.http11.Http11Protocol$Http11ConnectionHandler.process(Http11Protocol.java:583)
  at org.apache.tomcat.util.net.JIoEndpoint$SocketProcessor.run(JIoEndpoint.java:379)
  at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
  at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
  at java.lang.Thread.run(Thread.java:619)
Caused by: java.security.PrivilegedActionException: GSSException: Defective token detected (Mechanism level: GSSHeader did not find the right tag)
  at java.security.AccessController.doPrivileged(Native Method)
  at javax.security.auth.Subject.doAs(Subject.java:396)
  at org.springframework.security.extensions.kerberos.SunJaasKerberosTicketValidator.validateTicket(SunJaasKerberosTicketValidator.java:72)
  ... 25 more
Caused by: GSSException: Defective token detected (Mechanism level: GSSHeader did not find the right tag)
  at sun.security.jgss.GSSHeader.<init>(GSSHeader.java:80)
  at sun.security.jgss.GSSContextImpl.acceptSecContext(GSSContextImpl.java:287)
  at sun.security.jgss.GSSContextImpl.acceptSecContext(GSSContextImpl.java:267)
  at org.springframework.security.extensions.kerberos.SunJaasKerberosTicketValidator$KerberosValidateAction.run(SunJaasKerberosTicketValidator.java:161)
  at org.springframework.security.extensions.kerberos.SunJaasKerberosTicketValidator$KerberosValidateAction.run(SunJaasKerberosTicketValidator.java:1)
  ... 28 more
SecurityContextHolder now cleared, as request processing completed

It seems (from what we can make out) that the AD server sending an NTLM token (we can tell as it starts with "TlRMTVNT.....") to IE and IE is then sending this to our application and it's failing.

Our AD server should be sending a Kerberos/SPNEGO token to IE.

Other notes:

• Our server(tc server) and client(browser) are on different (virtual) machines and in same domain.

Answer 1

This can happen when you are running the client and server on the same machine. When you use IE to talk to the machine running tomcat ensure that these are distinct machines.

Additionally you need to ensure that the server machine is joined to the domain specified in the keytab (testdomain.ourcompany.co.uk) or you might drop back to NTLM. Your keytab can still work even if your server is on a machine not joined to the domain (you'll see the nice keytab decrypt that you showed), but IE can get confused and not do the correct thing.

AD only really likes to speak arcfour-hmac for Server 2003 so you need to ensure that you set this up correctly in your krb5.ini file.

You can correctly create the keytab like this:

C:\>ktpass -princ HTTP/ourweb.testdomain.ourcompany.co.uk@TESTDOMAIN.OURCOMPANY.CO.UK -mapuser ourweb.testdomain.ourcompany.co.uk@TESTDOMAIN.OURCOMPANY.CO.UK -crypto RC4-HMAC-NT -ptype K
RB5_NT_PRINCIPAL -pass * -out ourweb.keytab
Targeting domain controller: test-dc.ourcompany.co.uk
Using legacy password setting method
Successfully mapped HTTP/ourweb.testdomain.ourcompany.co.uk@TESTDOMAIN.OURCOMPANY.CO.UK to ourweb.testdomain.ourcompany.co.uk.
Key created.
Output keytab to ourweb.keytab:
Keytab version: 0x502
keysize 75 HTTP/ourweb.testdomain.ourcompany.co.uk@TESTDOMAIN.OURCOMPANY.CO.UK ptype 1 (KRB5_NT_PRINCIPAL)
vno 3 etype 0x17 (RC4-HMAC) keylength 16 (0x0fd0e500225c4fca9a63a9998b17ca32)

I did not see that you had set up a krb5.ini file. You will need to have that set correctly on your server machine (default location C:\\WINDOWS\\krb5.ini):

[domain_realm]  
    .testdomain.ourcompany.co.uk = TESTDOMAIN.OURCOMPANY.CO.UK
    testdomain.ourcompany.co.uk = TESTDOMAIN.OURCOMPANY.CO.UK

[libdefaults]   
    default_realm = TESTDOMAIN.OURCOMPANY.CO.UK
    permitted_enctypes = aes128-cts aes256-cts arcfour-hmac-md5 
    default_tgs_enctypes = aes128-cts aes256-cts arcfour-hmac-md5 
    default_tkt_enctypes = aes128-cts aes256-cts arcfour-hmac-md5 

[realms]    
VERDAD.LOCAL = {        
    kdc = test-dc.ourcompany.co.uk  
    admin_server = test-dc.ourcompany.co.uk
    default_domain = TESTDOMAIN.OURCOMPANY.CO.UK
}

You might also need to set the following properties (if you are trying to run this from an IDE):

<systemProperties>
  <java.security.krb5.kdc>test-dc.ourcompany.co.uk</java.security.krb5.kdc>
  <java.security.krb5.realm>TESTDOMAIN.OURCOMPANY.CO.UK</java.security.krb5.realm>
</systemProperties>

I was using the org.codehaus.mojo plugin for maven which sets these in the pom file like this:

<build>
  <plugins>
    <plugin>
      <groupId>org.codehaus.mojo</groupId>
      <artifactId>tomcat-maven-plugin</artifactId>
      <configuration>
        <server>tomcat-development-server</server>
        <port>8080</port>
        <path>/SecurityTest</path>
        <systemProperties>
          <java.security.krb5.kdc>test-dc.ourcompany.co.uk</java.security.krb5.kdc
          <java.security.krb5.realm>TESTDOMAIN.OURCOMPANY.CO.UK</java.security.krb5.realm>
        </systemProperties>
      </configuration>
    </plugin>
  </plugins>
</build>

Answer 2

I also encountered this problem. For those unlucky people who will have this problem in the future, another cause for this problem is accessing the server by ip instead of it's A record (hostname)

Answer 3

I also had same problem and took me very very long time to find the culprit. So if you have done all the above and still it uses NTLM token instead of kerberos. make sure you dont have duplicate SPN. in my case I had 2 accounts mapped to same SPN and the reason was I previously run a seperate web app on same server that used a different service account but mapped to same SPN which was HTTP/

Hope it helps

Answer 4

So in my case I set up two machines:

• Win-Root - 192.168.1.100 - My test domain controller
• Win-Child - 192.168.1.200 - Server that will run Tomcat service that will be secured by Kerberos

Then I installed Active directory domain services on Win-Root, and promoted to domain controller (Enables DNS). The name of the domain is Netbios= FUSIONIS , FQDN= fusionis.life .

Then I went into Win-Child and set the DNS server to 192.168.1.100, then joined the domain FUSIONIS .

Then I went into the Win-Root DNS settings and added a forward lookup zone for fusion.fusoinis.life .

Then I enabled Kerberos authentication to my tomcat app on http://fusion.fusionis.life:8080 .

I logged into Win-Child to test it out.

But I kept getting (Mechanism level: GSSHeader did not find the right tag) .

After an hour of head scratching, I realized... I accidentally was logged in as a local account! In other words, whoami returned "WIN-23523h253\Administrator". I totally forgot to actually log in as a domain account to the Win-Child server.

Then I logged into fusionis\testuser and everything worked as expected.

Lesson learned: Make sure you are not accidentally logged in as a non-domain account!