Apache Storm（1.2.1）傳輸異常管道中斷

Question

在最近升級的Storm 1.2.1版本中提交拓撲時，出現以下錯誤。

69001 [pool-7-thread-1] WARN  o.a.s.u.NimbusClient - Ignoring exception while trying to get leader nimbus info from <ip>. will retry with a different seed host.
java.lang.RuntimeException: java.lang.RuntimeException: org.apache.storm.thrift.transport.TTransportException: java.net.SocketException: Broken pipe
    at org.apache.storm.security.auth.ThriftClient.reconnect(ThriftClient.java:112) ~[storm-core-1.2.1.jar:1.2.1]
    at org.apache.storm.security.auth.ThriftClient.<init>(ThriftClient.java:73) ~[storm-core-1.2.1.jar:1.2.1]
    at org.apache.storm.utils.NimbusClient.<init>(NimbusClient.java:136) ~[storm-core-1.2.1.jar:1.2.1]
    at org.apache.storm.utils.NimbusClient.getConfiguredClientAs(NimbusClient.java:92) [storm-core-1.2.1.jar:1.2.1]
    at org.apache.storm.utils.NimbusClient.getConfiguredClientAs(NimbusClient.java:66) [storm-core-1.2.1.jar:1.2.1]
    at org.apache.storm.utils.NimbusClient.getConfiguredClient(NimbusClient.java:58) [storm-core-1.2.1.jar:1.2.1]
    at java.util.concurrent.FutureTask.run(FutureTask.java:262) [?:1.7.0_79]
    at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [?:1.7.0_79]
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [?:1.7.0_79]
    at java.lang.Thread.run(Thread.java:745) [?:1.7.0_79]
Caused by: java.lang.RuntimeException: org.apache.storm.thrift.transport.TTransportException: java.net.SocketException: Broken pipe
    at org.apache.storm.security.auth.TBackoffConnect.retryNext(TBackoffConnect.java:64) ~[storm-core-1.2.1.jar:1.2.1]
    at org.apache.storm.security.auth.TBackoffConnect.doConnectWithRetry(TBackoffConnect.java:56) ~[storm-core-1.2.1.jar:1.2.1]
    at org.apache.storm.security.auth.ThriftClient.reconnect(ThriftClient.java:104) ~[storm-core-1.2.1.jar:1.2.1]
    ... 16 more

檢查nimbus.log文件：

o.a.s.t.s.AbstractNonblockingServer$FrameBuffer Thread-9 [ERROR] Read a frame size of 16777216, which is bigger than the maximum allowable buffer size for ALL connections.

更新資料

更改了nimbus.thrift.max_buffer_size：20000000，並且拓撲未提交，客戶端和服務器中也沒有警告/錯誤日志。

殺死雨雲器時，我遇到了管道中斷的異常，因為傳輸可能不完整。 我不知道為什么這很慢。

造成此問題的原因是什么？

Answer 1

設置storm.thrift.transport
客戶端配置和storm.yml文件中的“ org.apache.storm.security.auth.plain.PlainSaslTransportPlugin”解決了該問題。

我在客戶端配置中的storm.yml和PlainSaslTransportPlugin中使用了SimpleTransportPlugin，這導致了此問題。