Cassandra Performance Tuning

Question

I have successfully installed a multi-node Cassandra cluster with 10nodes, The nodetool status command shows every node is UP and NORMAL.

but the Performance I am getting is very bad.

here are my results:

Operations /seconds = 4000
Read Latency = 13ms
write Latency = 10ms

I am using YCSB to measure performance

Tuning that I have done till now:

Consistency level = 1
Replication Factor = 3
Heap size = 4GB

My Hardware: Each node is a VM with CentOS

2GHZ CPU with 8 cores
8GB RAM
1GB/ps N/W

Please let me know what more settings I can tweak to get maximum performance out of my cluster.

Answer 1

If you have 1 system with 10 VMs running on it and 1 disk, the performance of any (not in-memory) database will be bad. Especially with spinning disks (no matter how expensive they are) is going to be a major contention point. With a really good SSD you may be able to pull off a few instances, but performance stress testing will likely always hit either that or a CPU bottleneck (if things configured correctly for system).

Pretty good chance with 4gb heaps and a stress workload you are going to be hitting GC and memory issues, do you have any monitoring around that? Can use visualvm and connect to the ip:7199 (ip set in cassandra-env.sh ).

8gb of ram per vm is on the minimum spec end. You want at least 8gb of JVM heap with space for the offheap stuff and OS. A 16gb system is likely sufficient. Once again the shared disk will kill performance so it will only go so far. but should be able to do far better than 4k/sec.