BootstrapMethodError - 找不到类定义

Question

I have a Java webapp running on Jetty (version 9.4.20). It uses a custom library for accessing MySql database, which has always worked well.

At random, sometimes, this exception is thrown:

java.lang.BootstrapMethodError: java.lang.NoClassDefFoundError: java/lang/IndyBootstrapDispatcher
        at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:222)
        at org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:844)
        at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1604)
        at org.eclipse.jetty.websocket.server.WebSocketUpgradeFilter.doFilter(WebSocketUpgradeFilter.java:226)
        at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1591)
        at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:542)
        at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143)
        at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:536)
        at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127)
        at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:235)
        at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:1581)
        at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:233)
        at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1307)
        at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:188)
        at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:482)
        at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:1549)
        at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:186)
        at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1204)
        at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141)
        at org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:221)
        at org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:146)
        at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127)
        at org.eclipse.jetty.server.Server.handle(Server.java:494)
        at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:374)
        at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:268)
        at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:311)
        at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:103)
        at org.eclipse.jetty.io.ChannelEndPoint$2.run(ChannelEndPoint.java:117)
        at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.runTask(EatWhatYouKill.java:336)
        at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:313)
        at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:171)
        at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.run(EatWhatYouKill.java:129)
        at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:367)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:782)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:918)
        at java.lang.Thread.run(Thread.java:748)
Caused by: java.lang.NoClassDefFoundError: java/lang/IndyBootstrapDispatcher
        ... 36 more
java.lang.BootstrapMethodError: java.lang.NoClassDefFoundError: java/lang/IndyBootstrapDispatcher
        at com.mysql.jdbc.StatementImpl.executeQuery(StatementImpl.java:1421)
        at com.mchange.v2.c3p0.impl.DefaultConnectionTester.activeCheckConnection(DefaultConnectionTester.java:286)
        at com.mchange.v2.c3p0.impl.C3P0PooledConnectionPool$1PooledConnectionResourcePoolManager.testPooledConnection(C3P0PooledConnectionPool.java:510)
        at com.mchange.v2.c3p0.impl.C3P0PooledConnectionPool$1PooledConnectionResourcePoolManager.refurbishResourceOnCheckin(C3P0PooledConnectionPool.java:406)
        at com.mchange.v2.resourcepool.BasicResourcePool.attemptRefurbishResourceOnCheckin(BasicResourcePool.java:1760)
        at com.mchange.v2.resourcepool.BasicResourcePool.access$200(BasicResourcePool.java:44)
        at com.mchange.v2.resourcepool.BasicResourcePool$1RefurbishCheckinResourceTask.run(BasicResourcePool.java:1378)
        at com.mchange.v2.async.ThreadPoolAsynchronousRunner$PoolThread.run(ThreadPoolAsynchronousRunner.java:696)
Caused by: java.lang.NoClassDefFoundError: java/lang/IndyBootstrapDispatcher

which freezes the service.

Strange thing is that everything was working and no changes to this library (dependencies, code, ...) or other updates were made.

JDK used is 1.8.0_201.

Now, I know how to handle this kind of exception, but I have not found any reference to java lang IndyBootstrapDispatcher by crawling sources of the interested libraries.

Any idea? Thank you all.

Answer 1

Jetty is starting with system temp directory for the Java system property java.io.tmpdir .

This is typically the system /tmp directory.

Jetty unpacks wars and accesses deep resources from your webapp from this temp directory.

What happens on some Linux distributions is that the the system temp directory ( /tmp ) is periodically cleaned up. This can break a running Jetty instance as the content it needs has now been removed out from underneath it.

The key is to use a temp directory for Jetty that isn't the system temp directory.

You can change the temp directory that Jetty uses in a number of ways.

See: Jetty: How to change startup temp directory

Answer 2

Answering my own question if someone gets here.

This is due to ELK apm connector (version used 1.18.0 RC 1 in auto attach mode); by disabling the agent the errors go away. Further tests will be done on different agent versions.

Relevant GitHub issues here and here .