cassandra drop keyspace hangs forever

Question

I try to drop a keyspace using cqlsh:

drop keyspace test;

It then hangs indefinitely. When I finally interrupt it, I can no longer use cqlsh. For example, this hangs too:

describe keyspaces;

I then have to restart cqlsh. When describing the keyspace, I have this:

cqlsh> describe keyspace test;

CREATE KEYSPACE test WITH replication = {
  'class': 'SimpleStrategy',
  'replication_factor': '1'
};
Bad Request: unconfigured table schema_columnfamilies

In /var/log/cassandra/system.log , all I have is this:

INFO  [Thrift:12] 2016-03-30 12:20:27,145 MigrationManager.java:448 - Drop Keyspace 'test'

I tried:

❱❱❱ sstablescrub keyspace test                                            
WARN  10:27:00 Small commitlog volume detected at /var/lib/cassandra/commitlog; setting commitlog_total_space_in_mb to 2314.  You can override this in cassandra.yaml
WARN  10:27:00 Only 4248 MB free across all data volumes. Consider adding more capacity to your cluster or removing obsolete snapshots
ERROR 10:27:01 Exiting due to error while processing commit log during initialization.
org.apache.cassandra.io.FSWriteError: java.nio.file.AccessDeniedException: /var/lib/cassandra/commitlog/CommitLog-6-1459333621738.log
    at org.apache.cassandra.db.commitlog.CommitLogSegment.<init>(CommitLogSegment.java:149) ~[apache-cassandra-3.0.4.jar:3.0.4]
    at org.apache.cassandra.db.commitlog.MemoryMappedSegment.<init>(MemoryMappedSegment.java:47) ~[apache-cassandra-3.0.4.jar:3.0.4]
    at org.apache.cassandra.db.commitlog.CommitLogSegment.createSegment(CommitLogSegment.java:122) ~[apache-cassandra-3.0.4.jar:3.0.4]
    at org.apache.cassandra.db.commitlog.CommitLogSegmentManager$1.runMayThrow(CommitLogSegmentManager.java:121) ~[apache-cassandra-3.0.4.jar:3.0.4]
    at org.apache.cassandra.utils.WrappedRunnable.run(WrappedRunnable.java:28) [apache-cassandra-3.0.4.jar:3.0.4]
    at java.lang.Thread.run(Thread.java:745) [na:1.8.0_73]
Caused by: java.nio.file.AccessDeniedException: /var/lib/cassandra/commitlog/CommitLog-6-1459333621738.log
    at sun.nio.fs.UnixException.translateToIOException(UnixException.java:84) ~[na:1.8.0_73]
    at sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:102) ~[na:1.8.0_73]
    at sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:107) ~[na:1.8.0_73]
    at sun.nio.fs.UnixFileSystemProvider.newFileChannel(UnixFileSystemProvider.java:177) ~[na:1.8.0_73]
    at java.nio.channels.FileChannel.open(FileChannel.java:287) ~[na:1.8.0_73]
    at java.nio.channels.FileChannel.open(FileChannel.java:335) ~[na:1.8.0_73]
    at org.apache.cassandra.db.commitlog.CommitLogSegment.<init>(CommitLogSegment.java:144) ~[apache-cassandra-3.0.4.jar:3.0.4]
    ... 5 common frames omitted

And also this:

nodetool scrub test

And this:

❱❱❱ nodetool repair test
[2016-03-30 12:28:44,970] Nothing to repair for keyspace 'test'

And this:

❱❱❱ nodetool clearsnapshot
Requested clearing snapshot(s) for [all keyspaces]

But nothing seems to work. Any idea how I could fix it ?

Answer 1

Happened to me also - due to inconsistency between the Cassandra server version and the Cassandra client version (CQLSH).

You're using cqlsh, so make sure the version of the cqlsh you're running is the same as the Cassandra you're connecting to.