[英]Turning cassandra inter-node encryption on causes “Unable to gossip with any seeds” exception
I am trying to turn cassandra (2.1) inter-node encryption on. 我正在尝试打开cassandra(2.1)节点间加密。 For testing purposes I am trying to start a 2 node cluster.
为了进行测试,我尝试启动一个2节点群集。
I am running each node inside a docker container on 2 separate ec2 instances. 我正在2个单独的ec2实例上的docker容器中运行每个节点。 Without inter-node encryption, everything works as expected.
没有节点间加密,一切都会按预期进行。
I am generating the ssl keys using the following script (taken from https://docs.jboss.org/author/display/RHQ/Cassandra+Node+To+Node+Encryption?_sscc=t ): 我正在使用以下脚本生成ssl密钥(取自https://docs.jboss.org/author/display/RHQ/Cassandra+Node+To+Node+Encryption?_sscc=t ):
for ((a=0; a < NUMBER_OF_NODES ; a++))
do
node_id=node${a}
echo -e "Start building certificates for ${node_id}"
echo -e "=========================================="
rm -vf ./${node_id}.keystore
rm -vf ./${node_id}.cer
#1 Generate key and store
${java_folder}/keytool -genkey -v -keyalg RSA -keysize 1024 -alias ${node_id} -keystore ${node_id}.keystore -storepass "${node_id}store" -dname 'CN=RHQ' -keypass "${node_id}store" -validity 3650
#2 Extract public certificate
${java_folder}/keytool -export -v -alias ${node_id} -file ${node_id}.cer -keystore ${node_id}.keystore -storepass "${node_id}store"
#3 Add public certificate to global keystore
${java_folder}/keytool -import -v -trustcacerts -alias ${node_id} -file ${node_id}.cer -keystore global.truststore -storepass 'globalstore' -noprompt
echo -e "========================================="
echo -e "Done building certificates for ${node_id}\n\n"
done
I am also adding the following configuration to each node's cassandra.yml file ( node0
changes accordingly): 我还将以下配置添加到每个节点的cassandra.yml文件中(
node0
更改):
server_encryption_options:
internode_encryption: all
keystore: /keystores/node0.keystore
keystore_password: node0store
truststore: /keystores/global.truststore
truststore_password: globalstore
node1
is configured with node0
as it's seed. node1
被配置为node0
作为它的种子。 I start node0, and wait until it starts, I see no exceptions, everything works as expected. 我启动了node0,然后等待它启动,我没有看到异常,一切都按预期进行。 Then I start node1, which throws the following (only when the debug level is set to "trace"):
然后,我启动node1,它引发以下消息(仅当调试级别设置为“ trace”时):
TRACE 08:14:16 unable to connect to 172.12.1.11/172.12.1.11
javax.net.ssl.SSLException: Unrecognized SSL message, plaintext connection?
at sun.security.ssl.InputRecord.handleUnknownRecord(InputRecord.java:671) ~[na:1.7.0_65]
at sun.security.ssl.InputRecord.read(InputRecord.java:504) ~[na:1.7.0_65]
at sun.security.ssl.SSLSocketImpl.readRecord(SSLSocketImpl.java:927) ~[na:1.7.0_65]
at sun.security.ssl.SSLSocketImpl.performInitialHandshake(SSLSocketImpl.java:1312) ~[na:1.7.0_65]
at sun.security.ssl.SSLSocketImpl.writeRecord(SSLSocketImpl.java:702) ~[na:1.7.0_65]
at sun.security.ssl.AppOutputStream.write(AppOutputStream.java:122) ~[na:1.7.0_65]
at java.io.BufferedOutputStream.flushBuffer(BufferedOutputStream.java:82) ~[na:1.7.0_65]
at java.io.BufferedOutputStream.flush(BufferedOutputStream.java:140) ~[na:1.7.0_65]
at org.apache.cassandra.io.util.DataOutputStreamPlus.flush(DataOutputStreamPlus.java:55) ~[apache-cassandra-2.1.1.jar:2.1.1]
at org.apache.cassandra.net.OutboundTcpConnection.connect(OutboundTcpConnection.java:347) [apache-cassandra-2.1.1.jar:2.1.1]
at org.apache.cassandra.net.OutboundTcpConnection.run(OutboundTcpConnection.java:163) [apache-cassandra-2.1.1.jar:2.1.1]
TRACE 08:14:17 Expired 0 entries
TRACE 08:14:20 Expired 0 entries
TRACE 08:14:22 Expired 0 entries
TRACE 08:14:25 Expired 0 entries
TRACE 08:14:27 Expired 0 entries
TRACE 08:14:30 Expired 0 entries
TRACE 08:14:32 Expired 0 entries
DEBUG 08:14:34 Copy GC in 14ms. CMS Old Gen: 9537256 -> 14901648; Eden Space: 41943040 -> 0; Survivor Space: 5242872 -> 5242880
TRACE 08:14:35 Expired 0 entries
ERROR 08:14:37 Exception encountered during startup
java.lang.RuntimeException: Unable to gossip with any seeds
at org.apache.cassandra.gms.Gossiper.doShadowRound(Gossiper.java:1221) ~[apache-cassandra-2.1.1.jar:2.1.1]
at org.apache.cassandra.service.StorageService.checkForEndpointCollision(StorageService.java:457) ~[apache-cassandra-2.1.1.jar:2.1.1]
at org.apache.cassandra.service.StorageService.prepareToJoin(StorageService.java:700) ~[apache-cassandra-2.1.1.jar:2.1.1]
at org.apache.cassandra.service.StorageService.initServer(StorageService.java:637) ~[apache-cassandra-2.1.1.jar:2.1.1]
at org.apache.cassandra.service.StorageService.initServer(StorageService.java:529) ~[apache-cassandra-2.1.1.jar:2.1.1]
at org.apache.cassandra.service.CassandraDaemon.setup(CassandraDaemon.java:324) [apache-cassandra-2.1.1.jar:2.1.1]
at org.apache.cassandra.service.CassandraDaemon.activate(CassandraDaemon.java:443) [apache-cassandra-2.1.1.jar:2.1.1]
at org.apache.cassandra.service.CassandraDaemon.main(CassandraDaemon.java:532) [apache-cassandra-2.1.1.jar:2.1.1]
java.lang.RuntimeException: Unable to gossip with any seeds
at org.apache.cassandra.gms.Gossiper.doShadowRound(Gossiper.java:1221)
at org.apache.cassandra.service.StorageService.checkForEndpointCollision(StorageService.java:457)
at org.apache.cassandra.service.StorageService.prepareToJoin(StorageService.java:700)
at org.apache.cassandra.service.StorageService.initServer(StorageService.java:637)
at org.apache.cassandra.service.StorageService.initServer(StorageService.java:529)
at org.apache.cassandra.service.CassandraDaemon.setup(CassandraDaemon.java:324)
at org.apache.cassandra.service.CassandraDaemon.activate(CassandraDaemon.java:443)
at org.apache.cassandra.service.CassandraDaemon.main(CassandraDaemon.java:532)
Exception encountered during startup: Unable to gossip with any seeds
It is also worth noting that on node0
port 7001 is open and accessible by node1
. 还值得注意的是,在
node0
端口7001是开放的,并且可由node1
访问。
As usually the case, the problem was related to the environment configuration and not to the actual cassandra settings. 通常情况下,问题与环境配置有关,而不与实际的cassandra设置有关。
I am running cassandra instances isolated inside a docker containers on a coreos cluster. 我正在隔离在coreos群集上的docker容器内的cassandra实例。 I forgot that the default etcd ssl port and cassandra's default ssl inter-node communication port are both 7001.
我忘记了默认的etcd ssl端口和cassandra的默认ssl节点间通信端口都是7001。
When changing one of the systems to work with an alternative port number the issue was resolved. 当更改其中一个系统以使用备用端口号时,此问题已解决。 I think that the error message could be more clear (and won't require trace debug level).
我认为该错误消息可能更清晰(并且不需要跟踪调试级别)。 A clearer error message could save me some time from tracing the network packets for answers.
更清晰的错误消息可以节省我跟踪网络数据包的时间。
声明:本站的技术帖子网页,遵循CC BY-SA 4.0协议,如果您需要转载,请注明本站网址或者原文地址。任何问题请咨询:yoyou2525@163.com.