[英]TThreadPoolServer Clients org.apache.thrift.transport.TTransportException: java.net.SocketException: Broken pipe
I am encountering an issue when the thrift client is tested with high load. 当节俭客户端经过高负载测试时,我遇到一个问题。
Setup: Thrift Server is a TThreadPoolServer. 设置:Thrift Server是一个TThreadPoolServer。 It performs and operation which takes only 1 -5 ms.Thrift Client connects to the server with a new connection each time, calls the operation once and closes the connection.
它执行和操作仅需要1 -5毫秒。Thrift客户端每次使用新连接连接到服务器,一次调用该操作并关闭连接。
When the load is at 160 TPS, the client starts to throw the following exception 当负载为160 TPS时,客户端开始引发以下异常
org.apache.thrift.transport.TTransportException: java.net.SocketException: Broken pipe at org.apache.thrift.transport.TIOStreamTransport.flush(TIOStreamTransport.java:161) ~[libthrift-0.9.1.jar:0.9.1]
at org.apache.thrift.TServiceClient.sendBase(TServiceClient.java:65) ~[libthrift-0.9.1.jar:0.9.1]
at com.ebay.traffic.email.delstats.thrift.interfaces.GSCassandreServices$Client.send_insertRow(GSCassandreServices.java:84) ~[GSCassandreServices$Client.class:na]
at com.ebay.traffic.email.delstats.thrift.interfaces.GSCassandreServices$Client.insertRow(GSCassandreServices.java:75) ~[GSCassandreServices$Client.class:na]
at com.ebay.traffic.email.delstats.thrift.impl.client.SimpleThriftClient.insertRowViaCQL(SimpleThriftClient.java:90) ~[SimpleThriftClient.class:na]
at com.ebay.traffic.email.delstats.entity.EventsEntityQueryHandlerImpl.insertEvents(EventsEntityQueryHandlerImpl.java:97) [EventsEntityQueryHandlerImpl.class:na]
at com.ebay.app.raptor.delstats.resources.StatsServiceResource.registerEvents(StatsServiceResource.java:306) [StatsServiceResource.class:na]
at sun.reflect.GeneratedMethodAccessor104.invoke(Unknown Source) ~[na:na]
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[na:1.7.0]
at java.lang.reflect.Method.invoke(Method.java:602) ~[na:2.6 (08-23-2013)]
at com.sun.jersey.spi.container.JavaMethodInvokerFactory$1.invoke(JavaMethodInvokerFactory.java:60) [jersey-server-1.8.jar:1.8]
at com.sun.jersey.server.impl.model.method.dispatch.AbstractResourceMethodDispatchProvider$ResponseOutInvoker._dispatch(AbstractResourceMethodDispatchProvider.java:205) [jersey-server-1.8.jar:1.8]
at com.sun.jersey.server.impl.model.method.dispatch.ResourceJavaMethodDispatcher.dispatch(ResourceJavaMethodDispatcher.java:75) [jersey-server-1.8.jar:1.8]
at com.sun.jersey.server.impl.uri.rules.HttpMethodRule.accept(HttpMethodRule.java:288) [jersey-server-1.8.jar:1.8]
at com.sun.jersey.server.impl.uri.rules.RightHandPathRule.accept(RightHandPathRule.java:147) [jersey-server-1.8.jar:1.8]
at com.sun.jersey.server.impl.uri.rules.ResourceClassRule.accept(ResourceClassRule.java:108) [jersey-server-1.8.jar:1.8]
at com.sun.jersey.server.impl.uri.rules.RightHandPathRule.accept(RightHandPathRule.java:147) [jersey-server-1.8.jar:1.8]
at com.sun.jersey.server.impl.uri.rules.RootResourceClassesRule.accept(RootResourceClassesRule.java:84) [jersey-server-1.8.jar:1.8]
at com.sun.jersey.server.impl.application.WebApplicationImpl._handleRequest(WebApplicationImpl.java:1469) [jersey-server-1.8.jar:1.8]
at com.sun.jersey.server.impl.application.WebApplicationImpl._handleRequest(WebApplicationImpl.java:1400) [jersey-server-1.8.jar:1.8]
at com.sun.jersey.server.impl.application.WebApplicationImpl.handleRequest(WebApplicationImpl.java:1349) [jersey-server-1.8.jar:1.8]
at com.sun.jersey.server.impl.application.WebApplicationImpl.handleRequest(WebApplicationImpl.java:1339) [jersey-server-1.8.jar:1.8]
Please note that connections succeed mostly, but I lose around 10% total connections due to this error 请注意,连接通常会成功,但是由于此错误,我的连接总数将减少10%
I checked netstat on the server and the connections are getting closed for every single request(Not many open connections at a time) and also there are no exceptions/errors on the server at all. 我检查了服务器上的netstat,并且每个请求的连接都已关闭(一次打开的连接数不多),并且服务器上也没有异常/错误。 Is this a limitation on the thrift server?
这是旧服务器的限制吗?
Thanks, Gopi 谢谢Gopi
My load opened lot of connections on the server which was more that desired. 我的负载打开了服务器上的大量连接,这比期望的要多。 I could see more than 5000 sockets opened in the server side at a moment.
我可以看到服务器端同时打开了5000多个套接字。 As EJP suggested, the best way to do deal with this is to use a thread pooled client which worked from me.
正如EJP所建议的,解决此问题的最佳方法是使用从我这里工作的线程池客户端。 This restricts the connections to the server and the server was able to accept all the connections.
这限制了与服务器的连接,并且服务器能够接受所有连接。
Ways to reproduce the issue: 重现此问题的方法:
声明:本站的技术帖子网页,遵循CC BY-SA 4.0协议,如果您需要转载,请注明本站网址或者原文地址。任何问题请咨询:yoyou2525@163.com.