啟用行緩存不會提高cassandra的性能
[英]enable row cache doesnot improve performance in cassandra
問題描述
我在Cassandra中創建表，並啟用了行緩存，但是啟用它並不能提高查詢性能，查詢延遲根本沒有改變。
Cassandra版本：3.11.1配置：row_cache_size_in_mb：10240 row_cache_save_period：3600
table schema:
CREATE TABLE worldscope.test_10 (
year int,
rank int,
cycname text,
name text,
value_0 float,
value_1 float,
value_2 float,
value_3 float,
value_4 float,
value_5 float,
value_6 float,
value_7 float,
value_8 float,
value_9 float,
PRIMARY KEY (year, rank)
) WITH CLUSTERING ORDER BY (rank ASC)
AND bloom_filter_fp_chance = 0.01
AND caching = {'keys': 'ALL', 'rows_per_partition': 100000}
AND comment = ''
AND compaction = {'class': 'org.apache.cassandra.db.compaction.SizeTieredCompactionStrategy', 'max_threshold': '32', 'min_threshold': '4'}
AND compression = {'chunk_length_in_kb': '64', 'class': 'org.apache.cassandra.io.compress.LZ4Compressor'}
AND crc_check_chance = 1.0
AND dclocal_read_repair_chance = 0.1
AND default_time_to_live = 0
AND gc_grace_seconds = 0
AND max_index_interval = 2048
AND memtable_flush_period_in_ms = 0
AND min_index_interval = 128
AND read_repair_chance = 0.0
AND speculative_retry = '99PERCENTILE';
查詢：從test_10中選擇value_1，其中year = 2016
當我運行nodetool時，它顯示行緩存被命中，如下所示：
Key Cache : entries 144, size 103.85 KiB, capacity 100 MiB, 567 hits, 723 requests, 0.784 recent hit rate, 14400 save period in seconds
Row Cache : entries 31, size 2.07 MiB, capacity 10 GiB, 279 hits, 310 requests, 0.900 recent hit rate, 3600 save period in seconds
但是無論是否啟用行緩存，查詢延遲都相同，並且在某些測試情況下，啟用行緩存后延遲甚至更長。
運行跟蹤時，詳細信息如下：啟用行緩存時：
Tracing session: 0667b5d0-1c33-11e8-8e0d-c18210ebaedc
activity | timestamp | source | source_elapsed | client
---------------------------------------------------------------------------------------------------+----------------------------+-------------+----------------+-------------
Execute CQL3 query | 2018-02-27 21:56:51.117000 | 10.81.220.6 | 0 | 10.81.220.6
Parsing select value_0 from test_10 where year = 20160102 limit 10; [Native-Transport-Requests-1] | 2018-02-27 21:56:51.117000 | 10.81.220.6 | 87 | 10.81.220.6
Preparing statement [Native-Transport-Requests-1] | 2018-02-27 21:56:51.117000 | 10.81.220.6 | 154 | 10.81.220.6
Read-repair DC_LOCAL [Native-Transport-Requests-1] | 2018-02-27 21:56:51.117000 | 10.81.220.6 | 284 | 10.81.220.6
Row cache hit [ReadStage-3] | 2018-02-27 21:56:51.119000 | 10.81.220.6 | 2366 | 10.81.220.6
Read 10 live and 0 tombstone cells [ReadStage-3] | 2018-02-27 21:56:51.119000 | 10.81.220.6 | 2460 | 10.81.220.6
Request complete | 2018-02-27 21:56:51.119563 | 10.81.220.6 | 2563 | 10.81.220.6
未啟用行緩存時：跟蹤會話：bc0cd420-1c32-11e8-8e0d-c18210ebaedc
activity | timestamp | source | source_elapsed | client
---------------------------------------------------------------------------------------------------+----------------------------+-------------+----------------+-------------
Execute CQL3 query | 2018-02-27 21:54:46.370000 | 10.81.220.6 | 0 | 10.81.220.6
Parsing select value_0 from test_10 where year = 20160102 limit 10; [Native-Transport-Requests-1] | 2018-02-27 21:54:46.370000 | 10.81.220.6 | 124 | 10.81.220.6
Preparing statement [Native-Transport-Requests-1] | 2018-02-27 21:54:46.370000 | 10.81.220.6 | 230 | 10.81.220.6
Executing single-partition query on test_10 [ReadStage-3] | 2018-02-27 21:54:46.370000 | 10.81.220.6 | 570 | 10.81.220.6
Acquiring sstable references [ReadStage-3] | 2018-02-27 21:54:46.370000 | 10.81.220.6 | 610 | 10.81.220.6
Skipped 0/1 non-slice-intersecting sstables, included 0 due to tombstones [ReadStage-3] | 2018-02-27 21:54:46.370000 | 10.81.220.6 | 639 | 10.81.220.6
Key cache hit for sstable 1 [ReadStage-3] | 2018-02-27 21:54:46.370001 | 10.81.220.6 | 690 | 10.81.220.6
Merged data from memtables and 1 sstables [ReadStage-3] | 2018-02-27 21:54:46.371000 | 10.81.220.6 | 847 | 10.81.220.6
Read 10 live and 0 tombstone cells [ReadStage-3] | 2018-02-27 21:54:46.371000 | 10.81.220.6 | 878 | 10.81.220.6
Request complete | 2018-02-27 21:54:46.370998 | 10.81.220.6 | 998 | 10.81.220.6
您會看到，未啟用緩存（998）顯然比啟用緩存（2563）快。
什么原因？？
2 個解決方案
解決方案1
3 2018-02-26 09:46:34
您是否使用TRACING運行了一些查詢，並可以發布結果？ （請參閱https://docs.datastax.com/en/cql/3.3/cql/cql_reference/cqlshTracing.html ）
當緩存命中會阻止磁盤讀取時，Cassandra將受益於行緩存。 您的緩存的命中率為0.9，這已經很高。 但是由於行緩存的大小只有大約2 MB的數據，我想您之前的讀取是從cassandras memtables提供的，而不是磁盤讀取的，因此加速效果非常小。
cqlsh:demo> SELECT value_1 FROM test_10 WHERE year = 2018;
value_1
---------
0.1
0.1
0.1
(3 rows)
Tracing session: 8ea1bd70-1ad9-11e8-b37d-63be3c2a5a29
activity | timestamp | source | source_elapsed
-------------------------------------------------------------------------------------------------+----------------------------+--------------+----------------
Execute CQL3 query | 2018-02-26 09:43:53.799000 | 85.10.240.52 | 0
Parsing SELECT value_1 FROM test_10 WHERE year = 2018; [SharedPool-Worker-1] | 2018-02-26 09:43:53.800000 | 85.10.240.52 | 262
Preparing statement [SharedPool-Worker-1] | 2018-02-26 09:43:53.800000 | 85.10.240.52 | 385
reading data from /85.10.240.60 [SharedPool-Worker-1] | 2018-02-26 09:43:53.800000 | 85.10.240.52 | 739
Sending READ message to /85.10.240.60 [MessagingService-Outgoing-/85.10.240.60] | 2018-02-26 09:43:53.800000 | 85.10.240.52 | 1005
READ message received from /85.10.240.52 [MessagingService-Incoming-/85.10.240.52] | 2018-02-26 09:43:53.802000 | 85.10.240.60 | 19
Executing single-partition query on test_10 [SharedPool-Worker-1] | 2018-02-26 09:43:53.802000 | 85.10.240.60 | 184
Acquiring sstable references [SharedPool-Worker-1] | 2018-02-26 09:43:53.802000 | 85.10.240.60 | 278
Merging memtable tombstones [SharedPool-Worker-1] | 2018-02-26 09:43:53.802000 | 85.10.240.60 | 323
Skipped 0/0 non-slice-intersecting sstables, included 0 due to tombstones [SharedPool-Worker-1] | 2018-02-26 09:43:53.802000 | 85.10.240.60 | 373
Merging data from memtables and 0 sstables [SharedPool-Worker-1] | 2018-02-26 09:43:53.802000 | 85.10.240.60 | 401
Read 3 live and 0 tombstone cells [SharedPool-Worker-1] | 2018-02-26 09:43:53.802001 | 85.10.240.60 | 608
Enqueuing response to /85.10.240.52 [SharedPool-Worker-1] | 2018-02-26 09:43:53.803000 | 85.10.240.60 | 709
REQUEST_RESPONSE message received from /85.10.240.60 [MessagingService-Incoming-/85.10.240.60] | 2018-02-26 09:43:53.803000 | 85.10.240.52 | 3972
Sending REQUEST_RESPONSE message to /85.10.240.52 [MessagingService-Outgoing-/85.10.240.52] | 2018-02-26 09:43:53.803000 | 85.10.240.60 | 948
Processing response from /85.10.240.60 [SharedPool-Worker-2] | 2018-02-26 09:43:53.804000 | 85.10.240.52 | 4060
Request complete | 2018-02-26 09:43:53.803553 | 85.10.240.52 | 4553
cqlsh:demo>
相關部分是：
Skipped 0/0 non-slice-intersecting sstables, included 0 due to tombstones [SharedPool-Worker-1] | 2018-02-26 09:43:53.802000 | 85.10.240.60 | 373
Merging data from memtables and 0 sstables [SharedPool-Worker-1] | 2018-02-26 09:43:53.802000 | 85.10.240.60 | 401
在我的小型演示案例中，沒有任何穩定的讀取或寫入操作。
解決方案2
0 2018-02-26 14:09:05
如果關閉行緩存，則似乎讀取sstable。 當行緩存打開時，跟蹤如下：
從test_10中選擇value_1，其中year = 20160102
Tracing session: 71705c70-1afc-11e8-892f-edfe3d6c5915
activity | timestamp | source | source_elapsed | client
------------------------------------------------------------------------------------------+----------------------------+-------------+----------------+-------------
Execute CQL3 query | 2018-02-26 08:53:37.208000 | 10.81.220.6 | 0 | 10.81.220.6
Parsing select value_1 from test_10 where year = 20160102; [Native-Transport-Requests-1] | 2018-02-26 08:53:37.208000 | 10.81.220.6 | 318 | 10.81.220.6
Preparing statement [Native-Transport-Requests-1] | 2018-02-26 08:53:37.208000 | 10.81.220.6 | 762 | 10.81.220.6
Row cache hit [ReadStage-2] | 2018-02-26 08:53:37.211000 | 10.81.220.6 | 3433 | 10.81.220.6
Read 100 live and 0 tombstone cells [ReadStage-2] | 2018-02-26 08:53:37.211000 | 10.81.220.6 | 3955 | 10.81.220.6
Request complete | 2018-02-26 08:53:37.223503 | 10.81.220.6 | 15503 | 10.81.220.6
Tracing session: 722ad2d0-1afc-11e8-892f-edfe3d6c5915
activity | timestamp | source | source_elapsed | client
------------------------------------------------------------------------------------------+----------------------------+-------------+----------------+-------------
Execute CQL3 query | 2018-02-26 08:53:38.429000 | 10.81.220.6 | 0 | 10.81.220.6
Parsing select value_1 from test_10 where year = 20160102; [Native-Transport-Requests-1] | 2018-02-26 08:53:38.429000 | 10.81.220.6 | 381 | 10.81.220.6
Preparing statement [Native-Transport-Requests-1] | 2018-02-26 08:53:38.429000 | 10.81.220.6 | 592 | 10.81.220.6
Row cache hit [ReadStage-2] | 2018-02-26 08:53:38.437000 | 10.81.220.6 | 8411 | 10.81.220.6
Read 100 live and 0 tombstone cells [ReadStage-2] | 2018-02-26 08:53:38.438000 | 10.81.220.6 | 8720 | 10.81.220.6
Request complete | 2018-02-26 08:53:38.449057 | 10.81.220.6 | 20057 | 10.81.220.6
當緩存關閉時：
Tracing session: efaad070-1afc-11e8-bafb-efaf5c6d9339
activity | timestamp | source | source_elapsed | client
------------------------------------------------------------------------------------------+----------------------------+-------------+----------------+-------------
Execute CQL3 query | 2018-02-26 08:57:08.984000 | 10.81.220.6 | 0 | 10.81.220.6
Parsing select value_1 from test_10 where year = 20160102; [Native-Transport-Requests-1] | 2018-02-26 08:57:08.988000 | 10.81.220.6 | 4448 | 10.81.220.6
Preparing statement [Native-Transport-Requests-1] | 2018-02-26 08:57:08.989000 | 10.81.220.6 | 4730 | 10.81.220.6
Executing single-partition query on test_10 [ReadStage-3] | 2018-02-26 08:57:08.993000 | 10.81.220.6 | 9314 | 10.81.220.6
Acquiring sstable references [ReadStage-3] | 2018-02-26 08:57:08.993000 | 10.81.220.6 | 9493 | 10.81.220.6
Skipped 0/1 non-slice-intersecting sstables, included 0 due to tombstones [ReadStage-3] | 2018-02-26 08:57:08.994000 | 10.81.220.6 | 9727 | 10.81.220.6
Partition index with 2 entries found for sstable 1 [ReadStage-3] | 2018-02-26 08:57:08.994000 | 10.81.220.6 | 10200 | 10.81.220.6
Merged data from memtables and 1 sstables [ReadStage-3] | 2018-02-26 08:57:08.997000 | 10.81.220.6 | 13002 | 10.81.220.6
Read 100 live and 0 tombstone cells [ReadStage-3] | 2018-02-26 08:57:08.997000 | 10.81.220.6 | 13265 | 10.81.220.6
Request complete | 2018-02-26 08:57:08.998950 | 10.81.220.6 | 14950 | 10.81.220.6
Tracing session: f0ad9a70-1afc-11e8-bafb-efaf5c6d9339
activity | timestamp | source | source_elapsed | client
------------------------------------------------------------------------------------------+----------------------------+-------------+----------------+-------------
Execute CQL3 query | 2018-02-26 08:57:10.679000 | 10.81.220.6 | 0 | 10.81.220.6
Parsing select value_1 from test_10 where year = 20160102; [Native-Transport-Requests-1] | 2018-02-26 08:57:10.679000 | 10.81.220.6 | 355 | 10.81.220.6
Preparing statement [Native-Transport-Requests-1] | 2018-02-26 08:57:10.680000 | 10.81.220.6 | 752 | 10.81.220.6
Executing single-partition query on test_10 [ReadStage-2] | 2018-02-26 08:57:10.682000 | 10.81.220.6 | 3110 | 10.81.220.6
Acquiring sstable references [ReadStage-2] | 2018-02-26 08:57:10.682000 | 10.81.220.6 | 3339 | 10.81.220.6
Skipped 0/1 non-slice-intersecting sstables, included 0 due to tombstones [ReadStage-2] | 2018-02-26 08:57:10.682000 | 10.81.220.6 | 3477 | 10.81.220.6
Key cache hit for sstable 1 [ReadStage-2] | 2018-02-26 08:57:10.683000 | 10.81.220.6 | 3598 | 10.81.220.6
Merged data from memtables and 1 sstables [ReadStage-2] | 2018-02-26 08:57:10.684000 | 10.81.220.6 | 5438 | 10.81.220.6
Read 100 live and 0 tombstone cells [ReadStage-2] | 2018-02-26 08:57:10.685000 | 10.81.220.6 | 5593 | 10.81.220.6
Request complete | 2018-02-26 08:57:10.685204 | 10.81.220.6 | 6204 | 10.81.220.6
通過使用Redis作為緩存，可以提高HBase的讀取性能嗎？
[英]Could I can improve HBase reading performance by using Redis as cache?
聲明:本站的技術帖子網頁，遵循CC BY-SA 4.0協議，如果您需要轉載，請注明本站網址或者原文地址。任何問題請咨詢:yoyou2525@163.com.