執行zookeeper shell時報錯zookeeper SSL

Question

我在 zookeeper 中配置了 ssl，當我看到日志 tls 運行正常時，但是當我運行 zookeeper shell 時突然出錯並退出這個日志

執行 zookeeper shell 后的此日志：[2022-11-08 03:47:16,761] WARN zookeeper.ssl.keyStore.location not specified (org.apache.zookeeper.common.X509Util) [2022-11-08 03:47: 16,762] WARN zookeeper.ssl.trustStore.location not specified (org.apache.zookeeper.common.X509Util) [2022-11-08 03:47:16,886] ERROR 與 session 08zokeeperokeeper 握手失敗 0xzo.302035 (org.apache.zookeeper.common.X509Util) NettyServerCnxnFactory) [2022-11-08 03:47:16,900] WARN 捕獲異常 (org.apache.zookeeper.server.NettyServerCnxnFactory) io.netty.handler.codec.DecoderException: io.netty.handler.ssl.NotSslRecordException: 不是SSL/TLS record: 0000002d000000000000000000000000000075300000000000000000000000100000000000000000000000000000000000 at io.netty.handler.codec.ByteToMessageDecoder.callDecode(ByteToMessageDecoder.java:480) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:279) at io.netty.c hannel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty .channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) 在 io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:166) 在 io.netty.channel.nio.netty.channel.channel.ioopSelectedKeyvented.io.netty NioEventLoop.java:722) 在 io.netty.ch annel.nio.NioEventLoop.processSelectedKeysOptimized(NioEventLoop.java:658) 在 io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:584) 在 io.netty.channel.nio.NioEventLoop.run(NioEventLoop.8:8924582.8) 496) 在 io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) 在 io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) 在 io.netty.util.concurrent .FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) at java.lang.Thread.run(Thread.java:750) Caused by: io.netty.handler.ssl.NotSslRecordException: not an SSL/TLS record: 0000002d000000000000000000000000000075300000000000000000000000100000000000000000000000000000000000 at io. netty.handler.ssl.SslHandler.decodeJdkCompatible(SslHandler.java:1215) 在 io.netty.handler.ssl.SslHandler.decode(SslHandler.java:1285) 在 B yteToMessageDecoder.decodeRemovalReentryProtection(ByteToMessageDecoder.java:510) at io.netty.handler.codec.ByteToMessageDecoder.callDecode(ByteToMessageDecoder.java:449)... 17 更多 [2022-11-08 08:47:47] session 0x0 (org.apache.zookeeper.server.NettyServerCnxnFactory) [2022-11-08 03:47:18,622] WARN 捕獲到異常 (org.apache.zookeeper.server.NettyServerCnxnFactory) io.io.netDeler.handlerException .netty.handler.ssl.NotSslRecordException: not an SSL/TLS record: 0000002d000000000000000000000000000075300000000000000000000000100000000000000000000000000000000000 at io.netty.handler.codec.ByteToMessageDecoder.callDecode(ByteToMessageDecoder.java:480) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder .java:279) 在 io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) 在 io.netty.ch annel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io .netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read（AbstractNioByteChannel.java:166）在io.netty.channel.nio.NioEventLoop.processSelectedKey（NioEventLoop.java:722）在io.netty.NiochannelE.noop processSelectedKeysOptimized(NioEventLoop.java:658) 在 io.netty.channel.nio.NioEventLo op.processSelectedKeys(NioEventLoop.java:584) 在 io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:496) 在 io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.8892132) 在 4:8692132 io.netty.util.internal.ThreadExecutorMap$2.run（ThreadExecutorMap.java:74）在io.netty.util.concurrent.FastThreadLocalRunnable.run（FastThreadLocalRunnable.java:30）在java.lang.8Thread.Thread.Thread. :750) Caused by: io.netty.handler.ssl.NotSslRecordException: not an SSL/TLS record: 0000002d000000000000000000000000000075300000000000000000000000100000000000000000000000000000000000 at io.netty.handler.ssl.SslHandler.decodeJdkCompatible(SslHandler.java:1215) at io.netty.handler. ssl.SslHandler.decode(SslHandler.java:1285) 在 io.netty.handler.codec.ByteToMessageDecoder.decodeRemovalReentryProtection(ByteToMessageDecoder.java:510) 在 io.netty.Bhandler.codec yteToMessageDecoder.callDecode(ByteToMessageDecoder.java:449)

this log before execute zookeeper sheell
[2022-11-08 03:44:09,105] WARN Send worker leaving thread id 3 my id = 2 (org.apache.zookeeper.server.quorum.QuorumCnxManager)
[2022-11-08 03:44:09,131] INFO Accepted TLS connection from /192.168.30.232:52204 - TLSv1.2 - TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA256 (org.apache.zookeeper.server.quorum.UnifiedServerSocket)
[2022-11-08 03:44:09,141] INFO Notification: my state:LOOKING; n.sid:3, n.state:LEADING, n.leader:3, n.round:0x12, n.peerEpoch:0x15, n.zxid:0x1400000002, message format version:0x2, n.config version:0x0 (org.apache.zookeeper.server.quorum.FastLeaderElection)
[2022-11-08 03:44:09,141] INFO Notification: my state:LOOKING; n.sid:3, n.state:LEADING, n.leader:3, n.round:0x12, n.peerEpoch:0x15, n.zxid:0x1400000002, message format version:0x2, n.config version:0x0 (org.apache.zookeeper.server.quorum.FastLeaderElection)
[2022-11-08 03:44:09,142] INFO Peer state changed: following (org.apache.zookeeper.server.quorum.QuorumPeer)
[2022-11-08 03:44:09,142] INFO FOLLOWING (org.apache.zookeeper.server.quorum.QuorumPeer)
[2022-11-08 03:44:09,148] INFO leaderConnectDelayDuringRetryMs: 100 (org.apache.zookeeper.server.quorum.Learner)
[2022-11-08 03:44:09,148] INFO TCP NoDelay set to: true (org.apache.zookeeper.server.quorum.Learner)
[2022-11-08 03:44:09,148] INFO zookeeper.learner.asyncSending = false (org.apache.zookeeper.server.quorum.Learner)
[2022-11-08 03:44:09,150] INFO Weighed connection throttling is disabled (org.apache.zookeeper.server.BlueThrottle)
[2022-11-08 03:44:09,151] INFO minSessionTimeout set to 4000 (org.apache.zookeeper.server.ZooKeeperServer)
[2022-11-08 03:44:09,153] INFO maxSessionTimeout set to 40000 (org.apache.zookeeper.server.ZooKeeperServer)
[2022-11-08 03:44:09,155] INFO Response cache size is initialized with value 400. (org.apache.zookeeper.server.ResponseCache)
[2022-11-08 03:44:09,155] INFO Response cache size is initialized with value 400. (org.apache.zookeeper.server.ResponseCache)
[2022-11-08 03:44:09,156] INFO zookeeper.pathStats.slotCapacity = 60 (org.apache.zookeeper.server.util.RequestPathMetricsCollector)
[2022-11-08 03:44:09,156] INFO zookeeper.pathStats.slotDuration = 15 (org.apache.zookeeper.server.util.RequestPathMetricsCollector)
[2022-11-08 03:44:09,156] INFO zookeeper.pathStats.maxDepth = 6 (org.apache.zookeeper.server.util.RequestPathMetricsCollector)
[2022-11-08 03:44:09,156] INFO zookeeper.pathStats.initialDelay = 5 (org.apache.zookeeper.server.util.RequestPathMetricsCollector)
[2022-11-08 03:44:09,156] INFO zookeeper.pathStats.delay = 5 (org.apache.zookeeper.server.util.RequestPathMetricsCollector)
[2022-11-08 03:44:09,156] INFO zookeeper.pathStats.enabled = false (org.apache.zookeeper.server.util.RequestPathMetricsCollector)
[2022-11-08 03:44:09,158] INFO The max bytes for all large requests are set to 104857600 (org.apache.zookeeper.server.ZooKeeperServer)
[2022-11-08 03:44:09,158] INFO The large request threshold is set to -1 (org.apache.zookeeper.server.ZooKeeperServer)
[2022-11-08 03:44:09,158] INFO Created server with tickTime 2000 minSessionTimeout 4000 maxSessionTimeout 40000 clientPortListenBacklog -1 datadir /var/log/zookeeper-logs/version-2 snapdir /var/log/zookeeper-logs/version-2 (org.apache.zookeeper.server.ZooKeeperServer)
[2022-11-08 03:44:09,159] INFO FOLLOWING - LEADER ELECTION TOOK - 575 MS (org.apache.zookeeper.server.quorum.Learner)
[2022-11-08 03:44:09,162] INFO Peer state changed: following - discovery (org.apache.zookeeper.server.quorum.QuorumPeer)
[2022-11-08 03:44:09,236] INFO Successfully connected to leader, using address: /192.168.30.232:2888 (org.apache.zookeeper.server.quorum.Learner)
[2022-11-08 03:44:09,273] INFO Peer state changed: following - synchronization (org.apache.zookeeper.server.quorum.QuorumPeer)
[2022-11-08 03:44:09,276] INFO Getting a diff from the leader 0x1400000002 (org.apache.zookeeper.server.quorum.Learner)
[2022-11-08 03:44:09,276] INFO Peer state changed: following - synchronization - diff (org.apache.zookeeper.server.quorum.QuorumPeer)
[2022-11-08 03:44:09,280] INFO Learner received NEWLEADER message (org.apache.zookeeper.server.quorum.Learner)
[2022-11-08 03:44:09,280] INFO Dynamic reconfig is disabled, we don't store the last seen config. (org.apache.zookeeper.server.quorum.QuorumPeer)
[2022-11-08 03:44:09,314] INFO Peer state changed: following - synchronization (org.apache.zookeeper.server.quorum.QuorumPeer)
[2022-11-08 03:44:09,320] INFO Configuring CommitProcessor with readBatchSize -1 commitBatchSize 1 (org.apache.zookeeper.server.quorum.CommitProcessor)
[2022-11-08 03:44:09,320] INFO Configuring CommitProcessor with 2 worker threads. (org.apache.zookeeper.server.quorum.CommitProcessor)
[2022-11-08 03:44:09,325] INFO zookeeper.request_throttler.shutdownTimeout = 10000 (org.apache.zookeeper.server.RequestThrottler)
[2022-11-08 03:44:09,342] INFO Learner received UPTODATE message (org.apache.zookeeper.server.quorum.Learner)
[2022-11-08 03:44:09,343] INFO Peer state changed: following - broadcast (org.apache.zookeeper.server.quorum.QuorumPeer)


i have change CA, keystore, trustore but this can't help me

Answer 1

您的 Zk SSL 設置有問題。 我的是這樣的，SSL 有效：

secureClientPort=12181
authProvider.x509=org.apache.zookeeper.server.auth.X509AuthenticationProvider
serverCnxnFactory=org.apache.zookeeper.server.NettyServerCnxnFactory
ssl.protocol=TLSv1.2
ssl.trustStore.location=XXX
ssl.trustStore.password=XXX
ssl.keyStore.location=XXX
ssl.keyStore.password=XXX
ssl.clientAuth=none

您可以在此處和此處找到更多詳細信息。