Cassandra Authentication Fail: "Unable to perform authentication: Cannot achieve consistency level QUORUM"

Question

I'm configuring a 3 node Cassandra cluster (multi datacenter) and everything works well until I set up the authentication process, setting from AllowAllAuthenticator to PasswordAuthenticator, as defined in Cassandra's doc.

The problem is, once I changed and restart nodes, I cannot access anymore the database, in this case with cassandra superuser, displaying this message:

Connection error: ('Unable to connect to any servers', {'10.0.0.10': AuthenticationFailed('Failed to authenticate to 10.0.0.10: Error from server: code=0100 [Bad credentials] message="Unable to perform authentication: Cannot achieve consistency level QUORUM"',)})

It's important to mention that before to set up the authenticator, I already updated the system_auth to NetworkTopologyStratety, setting up each node.

Also, without authentication all replications work fine, which means the cluster is running fine.

Does anyone have some idea about it? This is really driving me crazy, once I didn't find any reference about it.

All the best!

Answer 1

My guess is that you need to run repair on all of the nodes for the "system_auth", and if you're running DSE, ensure any keyspace that starts with "dse" that has "simple strategy" is updated to Network Topology Strategy with appropriate DC and RF settings - and run repair on each node for those as well.

That should solve your problem. My guess is that you created your users and then updated the keyspaces to use Network Topology. Once done, any new records will be be propagated correctly, but the existing records need repair to "fan them out" as it won't happen on its own.

-Jim