Current frame's stack size doesn't match stackmap

Question

We are upgrading spring from 4.3.20.Release to 5.3.20 and Hibernate version is 5.2.3.Final with Java 11 and Tomcat 9. All the changes are getting compiled and working well in local. But when deployed to server via jenkins they are failing with "VerifyError" during Bean Creation.

I am aware of "Xverify:none" option to pass over this, but wanted to fix it without Xverify:none solution.

Stack Trace:

org.springframework.beans.factory.BeanCreationException: Error creating bean with 
name 'appStatusController': Lookup method resolution failed; nested exception is 
java.lang.IllegalStateException: Failed to introspect Class 
[com.officehours.controller.AppStatusController] from ClassLoader 
[ParallelWebappClassLoader
     context: stage
     delegate: false
       ----------> Parent Classloader:
     java.net.URLClassLoader@2436ea2f] at org.springframework.beans.factory.annotation.AutowiredAnnotationBeanPostProcessor 
     .determineCandidateConstructors(AutowiredAnnotationBeanPostProcessor.java:289)
    ....
    .....
     
  Caused by: java.lang.VerifyError: Inconsistent stackmap frames at branch target 1285
  Exception Details:
  Location:    com/officehours/controller/AppStatusController.appOk(Ljavax/servlet/http/HttpServletRequest;Ljavax/servlet/http/HttpServletResponse;)Lorg/springframework/web/servlet/ModelAndView; @1282: goto

  Reason:
     Current frame's stack size doesn't match stackmap.
  Current Frame:
     bci: @1282
     flags: { }
     locals: { 'com/officehours/controller/AppStatusController', 
           'javax/servlet/http/HttpServletRequest', 
           'javax/servlet/http/HttpServletResponse', 
           'com/newrelic/agent/bridge/ExitTracer', integer, null, top, top, top, top, 
           top, top, 'javax/servlet/http/HttpServletResponse', 
           'javax/servlet/http/HttpServletRequest', 
           'com/officehours/controller/AppStatusController', integer, 
           'org/springframework/web/servlet/ModelAndView', top, top, top, top, top, 
           'java/lang/Throwable', 'javax/servlet/http/HttpServletResponse', 
           'javax/servlet/http/HttpServletRequest', 
           'com/officehours/controller/AppStatusController', 'java/io/File', 
           'java/io/LineNumberReader', top, top, 
           'org/springframework/web/servlet/ModelAndView', 
           'org/springframework/web/servlet/ModelAndView' }
    stack: { 'org/springframework/web/servlet/ModelAndView' }
 Stackmap Frame:
   bci: @1285
   flags: { }
   locals: { 'com/officehours/controller/AppStatusController', 
              'javax/servlet/http/HttpServletRequest', 
               'javax/servlet/http/HttpServletResponse', 
              'com/newrelic/agent/bridge/ExitTracer', integer, null, top, top, top, 
               top, top, top, 'javax/servlet/http/HttpServletResponse', 
               'javax/servlet/http/HttpServletRequest', 
               'com/officehours/controller/AppStatusController', integer, 
               'org/springframework/web/servlet/ModelAndView', top, top, top, top, 
               top, top, 'javax/servlet/http/HttpServletResponse', 
               'javax/servlet/http/HttpServletRequest', 
               'com/officehours/controller/AppStatusController', 'java/io/File', 
               'java/io/LineNumberReader', top, top, 
               'org/springframework/web/servlet/ModelAndView', 
               'org/springframework/web/servlet/ModelAndView' }
    stack: { 'org/springframework/web/servlet/ModelAndView','org/springframework/web/servlet/ModelAndView' }

Any suggestions on this would be highly appreciated as this is blocking me in moving further with other process (like production deployment)

Answer 1

Finally, I was able to get to the root of it.

This error was due to Double weaving of the code (1 which happens normally and 2 due to the newRelic instrumentation plugin which we used). This comes into the picture when our application uses "AspectJ" and tries to have spring version > 4.2.

When we were comparing the Local Vs Server config and the only difference we noticed was the Instrumentation plugins. We thought of removing these instrumentation plugins and seeing if that is helping us and when we removed the newRelic.jar we noticed that the application was up and running along with AspectJ code (Without Xverify In Place).

Upon checking it looks like a long open issue with newRelic and their team is working to identify it https://github.com/newrelic/newrelic-java-agent/issues/347

We implemented the Option-3 from there by modifying our newRelic.yml to include the following piece

class_transformer:
com.newrelic.instrumentation.jax-rs-1.0:
    enabled: false
com.newrelic.instrumentation.spring-4.3.0:
    enabled: false

Post this the app was up and running as expected. Hope this helps!