线程转储显示很多被阻止的进程

Question

我正在调试一个GWT Web应用程序，该应用程序运行了一段时间后已经耗尽了permgen空间，并且在VisualVM的线程转储中注意到有248个进程在等待锁定相同的StringBuffer，例如：

2013-02-27 10:56:30
Full thread dump Java HotSpot(TM) 64-Bit Server VM (23.3-b01 mixed mode):

"Attach Listener" daemon prio=10 tid=0x00007f3bdc00f000 nid=0x3bee waiting on condition [0x0000000000000000]
   java.lang.Thread.State: RUNNABLE

   Locked ownable synchronizers:
    - None

"1301161716@qtp-1875293347-251" prio=10 tid=0x00007f3ba8209800 nid=0x301f waiting for monitor entry [0x00007f3b6a15f000]
   java.lang.Thread.State: BLOCKED (on object monitor)
    at org.mortbay.log.StdErrLog.warn(StdErrLog.java:128)
    - waiting to lock <0x0000000087b405c8> (a java.lang.StringBuffer)
    at org.mortbay.jetty.handler.ContextHandler$SContext.log(ContextHandler.java:1424)
    at com.google.gwt.user.server.rpc.RPCServletUtils.writeResponseForUnexpectedFailure(RPCServletUtils.java:389)
    at com.google.gwt.user.server.rpc.AbstractRemoteServiceServlet.doUnexpectedFailure(AbstractRemoteServiceServlet.java:110)
    at com.google.gwt.user.server.rpc.AbstractRemoteServiceServlet.doPost(AbstractRemoteServiceServlet.java:67)
    at javax.servlet.http.HttpServlet.service(HttpServlet.java:727)
    at javax.servlet.http.HttpServlet.service(HttpServlet.java:820)
    at org.mortbay.jetty.servlet.ServletHolder.handle(ServletHolder.java:511)
    at org.mortbay.jetty.servlet.ServletHandler.handle(ServletHandler.java:401)
    at org.mortbay.jetty.security.SecurityHandler.handle(SecurityHandler.java:216)
    at org.mortbay.jetty.servlet.SessionHandler.handle(SessionHandler.java:182)
    at org.mortbay.jetty.handler.ContextHandler.handle(ContextHandler.java:766)
    at org.mortbay.jetty.webapp.WebAppContext.handle(WebAppContext.java:450)
    at org.mortbay.jetty.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:230)
    at org.mortbay.jetty.handler.HandlerCollection.handle(HandlerCollection.java:114)
    at org.mortbay.jetty.handler.HandlerWrapper.handle(HandlerWrapper.java:152)
    at org.mortbay.jetty.Server.handle(Server.java:326)
    at org.mortbay.jetty.HttpConnection.handleRequest(HttpConnection.java:542)
    at org.mortbay.jetty.HttpConnection$RequestHandler.content(HttpConnection.java:945)
    at org.mortbay.jetty.HttpParser.parseNext(HttpParser.java:756)
    at org.mortbay.jetty.HttpParser.parseAvailable(HttpParser.java:218)
    at org.mortbay.jetty.HttpConnection.handle(HttpConnection.java:404)
    at org.mortbay.io.nio.SelectChannelEndPoint.run(SelectChannelEndPoint.java:410)
    at org.mortbay.thread.QueuedThreadPool$PoolThread.run(QueuedThreadPool.java:582)

   Locked ownable synchronizers:
    - None

"182760474@qtp-1875293347-250" prio=10 tid=0x00007f3ba8207800 nid=0x2ff7 waiting for monitor entry [0x00007f3b6a260000]
   java.lang.Thread.State: BLOCKED (on object monitor)
    at org.mortbay.log.StdErrLog.warn(StdErrLog.java:128)
    - waiting to lock <0x0000000087b405c8> (a java.lang.StringBuffer)
    at org.mortbay.jetty.handler.ContextHandler$SContext.log(ContextHandler.java:1424)
    at com.google.gwt.user.server.rpc.RPCServletUtils.writeResponseForUnexpectedFailure(RPCServletUtils.java:389)
    at com.google.gwt.user.server.rpc.AbstractRemoteServiceServlet.doUnexpectedFailure(AbstractRemoteServiceServlet.java:110)
    at com.google.gwt.user.server.rpc.AbstractRemoteServiceServlet.doPost(AbstractRemoteServiceServlet.java:67)
    at javax.servlet.http.HttpServlet.service(HttpServlet.java:727)
    at javax.servlet.http.HttpServlet.service(HttpServlet.java:820)
    at org.mortbay.jetty.servlet.ServletHolder.handle(ServletHolder.java:511)
    at org.mortbay.jetty.servlet.ServletHandler.handle(ServletHandler.java:401)
    at org.mortbay.jetty.security.SecurityHandler.handle(SecurityHandler.java:216)
    at org.mortbay.jetty.servlet.SessionHandler.handle(SessionHandler.java:182)
    at org.mortbay.jetty.handler.ContextHandler.handle(ContextHandler.java:766)
    at org.mortbay.jetty.webapp.WebAppContext.handle(WebAppContext.java:450)
    at org.mortbay.jetty.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:230)
    at org.mortbay.jetty.handler.HandlerCollection.handle(HandlerCollection.java:114)
    at org.mortbay.jetty.handler.HandlerWrapper.handle(HandlerWrapper.java:152)
    at org.mortbay.jetty.Server.handle(Server.java:326)
    at org.mortbay.jetty.HttpConnection.handleRequest(HttpConnection.java:542)
    at org.mortbay.jetty.HttpConnection$RequestHandler.content(HttpConnection.java:945)
    at org.mortbay.jetty.HttpParser.parseNext(HttpParser.java:756)
    at org.mortbay.jetty.HttpParser.parseAvailable(HttpParser.java:218)
    at org.mortbay.jetty.HttpConnection.handle(HttpConnection.java:404)
    at org.mortbay.io.nio.SelectChannelEndPoint.run(SelectChannelEndPoint.java:410)
    at org.mortbay.thread.QueuedThreadPool$PoolThread.run(QueuedThreadPool.java:582)

   Locked ownable synchronizers:
    - None

"1468146634@qtp-1875293347-249" prio=10 tid=0x00007f3ba8205800 nid=0x2f89 waiting for monitor entry [0x00007f3b6a361000]
   java.lang.Thread.State: BLOCKED (on object monitor)
    at org.mortbay.log.StdErrLog.warn(StdErrLog.java:128)
    - waiting to lock <0x0000000087b405c8> (a java.lang.StringBuffer)
    at org.mortbay.jetty.handler.ContextHandler$SContext.log(ContextHandler.java:1424)
    at com.google.gwt.user.server.rpc.RPCServletUtils.writeResponseForUnexpectedFailure(RPCServletUtils.java:389)
    at com.google.gwt.user.server.rpc.AbstractRemoteServiceServlet.doUnexpectedFailure(AbstractRemoteServiceServlet.java:110)
    at com.google.gwt.user.server.rpc.AbstractRemoteServiceServlet.doPost(AbstractRemoteServiceServlet.java:67)
    at javax.servlet.http.HttpServlet.service(HttpServlet.java:727)
    at javax.servlet.http.HttpServlet.service(HttpServlet.java:820)
    at org.mortbay.jetty.servlet.ServletHolder.handle(ServletHolder.java:511)
    at org.mortbay.jetty.servlet.ServletHandler.handle(ServletHandler.java:401)
    at org.mortbay.jetty.security.SecurityHandler.handle(SecurityHandler.java:216)
    at org.mortbay.jetty.servlet.SessionHandler.handle(SessionHandler.java:182)
    at org.mortbay.jetty.handler.ContextHandler.handle(ContextHandler.java:766)
    at org.mortbay.jetty.webapp.WebAppContext.handle(WebAppContext.java:450)
    at org.mortbay.jetty.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:230)
    at org.mortbay.jetty.handler.HandlerCollection.handle(HandlerCollection.java:114)
    at org.mortbay.jetty.handler.HandlerWrapper.handle(HandlerWrapper.java:152)
    at org.mortbay.jetty.Server.handle(Server.java:326)
    at org.mortbay.jetty.HttpConnection.handleRequest(HttpConnection.java:542)
    at org.mortbay.jetty.HttpConnection$RequestHandler.content(HttpConnection.java:945)
    at org.mortbay.jetty.HttpParser.parseNext(HttpParser.java:756)
    at org.mortbay.jetty.HttpParser.parseAvailable(HttpParser.java:218)
    at org.mortbay.jetty.HttpConnection.handle(HttpConnection.java:404)
    at org.mortbay.io.nio.SelectChannelEndPoint.run(SelectChannelEndPoint.java:410)
    at org.mortbay.thread.QueuedThreadPool$PoolThread.run(QueuedThreadPool.java:582)

   Locked ownable synchronizers:
    - None

...etc...

这有问题吗？ 是什么原因造成的？ 该应用程序正在开发计算机上运行，​​我是唯一查看它的人。

Answer 1

简短的答案：您的应用程序中发生一些内部异常，您需要检查服务器日志（码头日志，访问日志，应用程序日志等）以找出根本原因。

---

分析：

StringBuffer在org.mortbay.log.StdErrLog定义为实例变量：

 public class StdErrLog implements Logger { // ... StringBuffer _buffer = new StringBuffer(); //... } 

它的用法包装在一个synchronized块中：

 public void warn(String msg,Object arg0, Object arg1) { String d=_dateCache.now(); int ms=_dateCache.lastMs(); synchronized(_buffer) { tag(d,ms,":WARN:"); format(msg,arg0,arg1); System.err.println(_buffer.toString()); } } 

因此，如果有多个线程试图调用此方法，则只有一个线程会通过，其他线程则必须等待（如您的线程堆栈中）。

我们可以看到此方法是从com.google.gwt.user.server.rpc.AbstractRemoteServiceServlet.doUnexpectedFailure()调用的，这意味着您的内部代码中有一个内部异常。 以下是从JavaDoc引用的此方法 ：

仅针对不是服务方法签名的一部分或由SecurityExceptions，SerializationExceptions或RPC框架内的其他失败导致的异常或错误调用此方法。

由于有很多线程阻塞了此方法，并且您是唯一访问服务器的客户端，因此我猜想您的客户端代码中可能存在一个错误，该错误将大量意外请求发送到服务器。 您需要检查服务器日志（码头日志，访问日志，应用程序日志等），以确定根本原因。