enable row cache doesnot improve performance in cassandra
Question
I create table in Cassandra, and row cache is enabled, but enable it does not improve the query performance, query latency is not changed at all.
Cassandra version: 3.11.1 config: row_cache_size_in_mb: 10240 row_cache_save_period: 3600
table schema:
CREATE TABLE worldscope.test_10 (
year int,
rank int,
cycname text,
name text,
value_0 float,
value_1 float,
value_2 float,
value_3 float,
value_4 float,
value_5 float,
value_6 float,
value_7 float,
value_8 float,
value_9 float,
PRIMARY KEY (year, rank)
) WITH CLUSTERING ORDER BY (rank ASC)
AND bloom_filter_fp_chance = 0.01
AND caching = {'keys': 'ALL', 'rows_per_partition': 100000}
AND comment = ''
AND compaction = {'class': 'org.apache.cassandra.db.compaction.SizeTieredCompactionStrategy', 'max_threshold': '32', 'min_threshold': '4'}
AND compression = {'chunk_length_in_kb': '64', 'class': 'org.apache.cassandra.io.compress.LZ4Compressor'}
AND crc_check_chance = 1.0
AND dclocal_read_repair_chance = 0.1
AND default_time_to_live = 0
AND gc_grace_seconds = 0
AND max_index_interval = 2048
AND memtable_flush_period_in_ms = 0
AND min_index_interval = 128
AND read_repair_chance = 0.0
AND speculative_retry = '99PERCENTILE';
query: select value_1 from test_10 where year = 2016
When I run nodetool, it shows row cache is hit, as below:
Key Cache : entries 144, size 103.85 KiB, capacity 100 MiB, 567 hits, 723 requests, 0.784 recent hit rate, 14400 save period in seconds
Row Cache : entries 31, size 2.07 MiB, capacity 10 GiB, 279 hits, 310 requests, 0.900 recent hit rate, 3600 save period in seconds
But the query latency is still the same whether enable row cache or not, and in some test case, latency even become longer with row cache enabled.
When I run trace, detail as below: when row cache is enabled:
Tracing session: 0667b5d0-1c33-11e8-8e0d-c18210ebaedc
activity | timestamp | source | source_elapsed | client
---------------------------------------------------------------------------------------------------+----------------------------+-------------+----------------+-------------
Execute CQL3 query | 2018-02-27 21:56:51.117000 | 10.81.220.6 | 0 | 10.81.220.6
Parsing select value_0 from test_10 where year = 20160102 limit 10; [Native-Transport-Requests-1] | 2018-02-27 21:56:51.117000 | 10.81.220.6 | 87 | 10.81.220.6
Preparing statement [Native-Transport-Requests-1] | 2018-02-27 21:56:51.117000 | 10.81.220.6 | 154 | 10.81.220.6
Read-repair DC_LOCAL [Native-Transport-Requests-1] | 2018-02-27 21:56:51.117000 | 10.81.220.6 | 284 | 10.81.220.6
Row cache hit [ReadStage-3] | 2018-02-27 21:56:51.119000 | 10.81.220.6 | 2366 | 10.81.220.6
Read 10 live and 0 tombstone cells [ReadStage-3] | 2018-02-27 21:56:51.119000 | 10.81.220.6 | 2460 | 10.81.220.6
Request complete | 2018-02-27 21:56:51.119563 | 10.81.220.6 | 2563 | 10.81.220.6
when row cache is not enabled: Tracing session: bc0cd420-1c32-11e8-8e0d-c18210ebaedc
activity | timestamp | source | source_elapsed | client
---------------------------------------------------------------------------------------------------+----------------------------+-------------+----------------+-------------
Execute CQL3 query | 2018-02-27 21:54:46.370000 | 10.81.220.6 | 0 | 10.81.220.6
Parsing select value_0 from test_10 where year = 20160102 limit 10; [Native-Transport-Requests-1] | 2018-02-27 21:54:46.370000 | 10.81.220.6 | 124 | 10.81.220.6
Preparing statement [Native-Transport-Requests-1] | 2018-02-27 21:54:46.370000 | 10.81.220.6 | 230 | 10.81.220.6
Executing single-partition query on test_10 [ReadStage-3] | 2018-02-27 21:54:46.370000 | 10.81.220.6 | 570 | 10.81.220.6
Acquiring sstable references [ReadStage-3] | 2018-02-27 21:54:46.370000 | 10.81.220.6 | 610 | 10.81.220.6
Skipped 0/1 non-slice-intersecting sstables, included 0 due to tombstones [ReadStage-3] | 2018-02-27 21:54:46.370000 | 10.81.220.6 | 639 | 10.81.220.6
Key cache hit for sstable 1 [ReadStage-3] | 2018-02-27 21:54:46.370001 | 10.81.220.6 | 690 | 10.81.220.6
Merged data from memtables and 1 sstables [ReadStage-3] | 2018-02-27 21:54:46.371000 | 10.81.220.6 | 847 | 10.81.220.6
Read 10 live and 0 tombstone cells [ReadStage-3] | 2018-02-27 21:54:46.371000 | 10.81.220.6 | 878 | 10.81.220.6
Request complete | 2018-02-27 21:54:46.370998 | 10.81.220.6 | 998 | 10.81.220.6
You see, cache is not enabled(998)is apparently faster than when cache is enabled(2563).
What's the reason??
2 answers
solution1
3 2018-02-26 09:46:34
Did you run some of your queries with TRACING and can you post the results? (see https://docs.datastax.com/en/cql/3.3/cql/cql_reference/cqlshTracing.html )
Cassandra will benefit from a row cache when the cache hit will prevent a disk read. Your cache has a hit rate of 0.9 which is quite high already. But as the size of the row cache is about 2 MBytes of data only I guess your reads before were served from cassandras memtables and not from disk reads, so the speedup effect is very small.
cqlsh:demo> SELECT value_1 FROM test_10 WHERE year = 2018;
value_1
---------
0.1
0.1
0.1
(3 rows)
Tracing session: 8ea1bd70-1ad9-11e8-b37d-63be3c2a5a29
activity | timestamp | source | source_elapsed
-------------------------------------------------------------------------------------------------+----------------------------+--------------+----------------
Execute CQL3 query | 2018-02-26 09:43:53.799000 | 85.10.240.52 | 0
Parsing SELECT value_1 FROM test_10 WHERE year = 2018; [SharedPool-Worker-1] | 2018-02-26 09:43:53.800000 | 85.10.240.52 | 262
Preparing statement [SharedPool-Worker-1] | 2018-02-26 09:43:53.800000 | 85.10.240.52 | 385
reading data from /85.10.240.60 [SharedPool-Worker-1] | 2018-02-26 09:43:53.800000 | 85.10.240.52 | 739
Sending READ message to /85.10.240.60 [MessagingService-Outgoing-/85.10.240.60] | 2018-02-26 09:43:53.800000 | 85.10.240.52 | 1005
READ message received from /85.10.240.52 [MessagingService-Incoming-/85.10.240.52] | 2018-02-26 09:43:53.802000 | 85.10.240.60 | 19
Executing single-partition query on test_10 [SharedPool-Worker-1] | 2018-02-26 09:43:53.802000 | 85.10.240.60 | 184
Acquiring sstable references [SharedPool-Worker-1] | 2018-02-26 09:43:53.802000 | 85.10.240.60 | 278
Merging memtable tombstones [SharedPool-Worker-1] | 2018-02-26 09:43:53.802000 | 85.10.240.60 | 323
Skipped 0/0 non-slice-intersecting sstables, included 0 due to tombstones [SharedPool-Worker-1] | 2018-02-26 09:43:53.802000 | 85.10.240.60 | 373
Merging data from memtables and 0 sstables [SharedPool-Worker-1] | 2018-02-26 09:43:53.802000 | 85.10.240.60 | 401
Read 3 live and 0 tombstone cells [SharedPool-Worker-1] | 2018-02-26 09:43:53.802001 | 85.10.240.60 | 608
Enqueuing response to /85.10.240.52 [SharedPool-Worker-1] | 2018-02-26 09:43:53.803000 | 85.10.240.60 | 709
REQUEST_RESPONSE message received from /85.10.240.60 [MessagingService-Incoming-/85.10.240.60] | 2018-02-26 09:43:53.803000 | 85.10.240.52 | 3972
Sending REQUEST_RESPONSE message to /85.10.240.52 [MessagingService-Outgoing-/85.10.240.52] | 2018-02-26 09:43:53.803000 | 85.10.240.60 | 948
Processing response from /85.10.240.60 [SharedPool-Worker-2] | 2018-02-26 09:43:53.804000 | 85.10.240.52 | 4060
Request complete | 2018-02-26 09:43:53.803553 | 85.10.240.52 | 4553
cqlsh:demo>
The relevant part is:
Skipped 0/0 non-slice-intersecting sstables, included 0 due to tombstones [SharedPool-Worker-1] | 2018-02-26 09:43:53.802000 | 85.10.240.60 | 373
Merging data from memtables and 0 sstables [SharedPool-Worker-1] | 2018-02-26 09:43:53.802000 | 85.10.240.60 | 401
No sstable read or writes involved in my small demo case.
solution2
0 2018-02-26 14:09:05
Seems sstable is read if row cache is off. when row cache is on, trace as below:
select value_1 from test_10 where year = 20160102
Tracing session: 71705c70-1afc-11e8-892f-edfe3d6c5915
activity | timestamp | source | source_elapsed | client
------------------------------------------------------------------------------------------+----------------------------+-------------+----------------+-------------
Execute CQL3 query | 2018-02-26 08:53:37.208000 | 10.81.220.6 | 0 | 10.81.220.6
Parsing select value_1 from test_10 where year = 20160102; [Native-Transport-Requests-1] | 2018-02-26 08:53:37.208000 | 10.81.220.6 | 318 | 10.81.220.6
Preparing statement [Native-Transport-Requests-1] | 2018-02-26 08:53:37.208000 | 10.81.220.6 | 762 | 10.81.220.6
Row cache hit [ReadStage-2] | 2018-02-26 08:53:37.211000 | 10.81.220.6 | 3433 | 10.81.220.6
Read 100 live and 0 tombstone cells [ReadStage-2] | 2018-02-26 08:53:37.211000 | 10.81.220.6 | 3955 | 10.81.220.6
Request complete | 2018-02-26 08:53:37.223503 | 10.81.220.6 | 15503 | 10.81.220.6
Tracing session: 722ad2d0-1afc-11e8-892f-edfe3d6c5915
activity | timestamp | source | source_elapsed | client
------------------------------------------------------------------------------------------+----------------------------+-------------+----------------+-------------
Execute CQL3 query | 2018-02-26 08:53:38.429000 | 10.81.220.6 | 0 | 10.81.220.6
Parsing select value_1 from test_10 where year = 20160102; [Native-Transport-Requests-1] | 2018-02-26 08:53:38.429000 | 10.81.220.6 | 381 | 10.81.220.6
Preparing statement [Native-Transport-Requests-1] | 2018-02-26 08:53:38.429000 | 10.81.220.6 | 592 | 10.81.220.6
Row cache hit [ReadStage-2] | 2018-02-26 08:53:38.437000 | 10.81.220.6 | 8411 | 10.81.220.6
Read 100 live and 0 tombstone cells [ReadStage-2] | 2018-02-26 08:53:38.438000 | 10.81.220.6 | 8720 | 10.81.220.6
Request complete | 2018-02-26 08:53:38.449057 | 10.81.220.6 | 20057 | 10.81.220.6
when cache is off:
Tracing session: efaad070-1afc-11e8-bafb-efaf5c6d9339
activity | timestamp | source | source_elapsed | client
------------------------------------------------------------------------------------------+----------------------------+-------------+----------------+-------------
Execute CQL3 query | 2018-02-26 08:57:08.984000 | 10.81.220.6 | 0 | 10.81.220.6
Parsing select value_1 from test_10 where year = 20160102; [Native-Transport-Requests-1] | 2018-02-26 08:57:08.988000 | 10.81.220.6 | 4448 | 10.81.220.6
Preparing statement [Native-Transport-Requests-1] | 2018-02-26 08:57:08.989000 | 10.81.220.6 | 4730 | 10.81.220.6
Executing single-partition query on test_10 [ReadStage-3] | 2018-02-26 08:57:08.993000 | 10.81.220.6 | 9314 | 10.81.220.6
Acquiring sstable references [ReadStage-3] | 2018-02-26 08:57:08.993000 | 10.81.220.6 | 9493 | 10.81.220.6
Skipped 0/1 non-slice-intersecting sstables, included 0 due to tombstones [ReadStage-3] | 2018-02-26 08:57:08.994000 | 10.81.220.6 | 9727 | 10.81.220.6
Partition index with 2 entries found for sstable 1 [ReadStage-3] | 2018-02-26 08:57:08.994000 | 10.81.220.6 | 10200 | 10.81.220.6
Merged data from memtables and 1 sstables [ReadStage-3] | 2018-02-26 08:57:08.997000 | 10.81.220.6 | 13002 | 10.81.220.6
Read 100 live and 0 tombstone cells [ReadStage-3] | 2018-02-26 08:57:08.997000 | 10.81.220.6 | 13265 | 10.81.220.6
Request complete | 2018-02-26 08:57:08.998950 | 10.81.220.6 | 14950 | 10.81.220.6
Tracing session: f0ad9a70-1afc-11e8-bafb-efaf5c6d9339
activity | timestamp | source | source_elapsed | client
------------------------------------------------------------------------------------------+----------------------------+-------------+----------------+-------------
Execute CQL3 query | 2018-02-26 08:57:10.679000 | 10.81.220.6 | 0 | 10.81.220.6
Parsing select value_1 from test_10 where year = 20160102; [Native-Transport-Requests-1] | 2018-02-26 08:57:10.679000 | 10.81.220.6 | 355 | 10.81.220.6
Preparing statement [Native-Transport-Requests-1] | 2018-02-26 08:57:10.680000 | 10.81.220.6 | 752 | 10.81.220.6
Executing single-partition query on test_10 [ReadStage-2] | 2018-02-26 08:57:10.682000 | 10.81.220.6 | 3110 | 10.81.220.6
Acquiring sstable references [ReadStage-2] | 2018-02-26 08:57:10.682000 | 10.81.220.6 | 3339 | 10.81.220.6
Skipped 0/1 non-slice-intersecting sstables, included 0 due to tombstones [ReadStage-2] | 2018-02-26 08:57:10.682000 | 10.81.220.6 | 3477 | 10.81.220.6
Key cache hit for sstable 1 [ReadStage-2] | 2018-02-26 08:57:10.683000 | 10.81.220.6 | 3598 | 10.81.220.6
Merged data from memtables and 1 sstables [ReadStage-2] | 2018-02-26 08:57:10.684000 | 10.81.220.6 | 5438 | 10.81.220.6
Read 100 live and 0 tombstone cells [ReadStage-2] | 2018-02-26 08:57:10.685000 | 10.81.220.6 | 5593 | 10.81.220.6
Request complete | 2018-02-26 08:57:10.685204 | 10.81.220.6 | 6204 | 10.81.220.6
The technical post webpages of this site follow the CC BY-SA 4.0 protocol. If you need to reprint, please indicate the site URL or the original address.Any question please contact:yoyou2525@163.com.
Related Question
Enable the query cache in postgreSQL to improve performance
cassandra and row cache update
Cache resource files to improve performance
Does cache align improve performance?
How to improve cache performance on this MIPS code
How could cache improve the performance of a pipeline processor?
How to cache my javascript files to improve performance
hibernate second level cache with Redis -will it improve performance?
What type of cache better to use to improve performance of website?
Could I can improve HBase reading performance by using Redis as cache?
Related Tags