堆栈内存溢出
  繁体   English   中英

启用行缓存不会提高cassandra的性能

[英]enable row cache doesnot improve performance in cassandra

 原文  2018-02-26 08:20:35  2   2    caching/ cassandra/ row

问题描述

我在Cassandra中创建表，并启用了行缓存，但是启用它并不能提高查询性能，查询延迟根本没有改变。

Cassandra版本：3.11.1配置：row_cache_size_in_mb：10240 row_cache_save_period：3600 

table schema:
CREATE TABLE worldscope.test_10 (
year int,
rank int,
cycname text,
name text,
value_0 float,
value_1 float,
value_2 float,
value_3 float,
value_4 float,
value_5 float,
value_6 float,
value_7 float,
value_8 float,
value_9 float,
PRIMARY KEY (year, rank)
) WITH CLUSTERING ORDER BY (rank ASC)
AND bloom_filter_fp_chance = 0.01
AND caching = {'keys': 'ALL', 'rows_per_partition': 100000}
AND comment = ''
AND compaction = {'class':    'org.apache.cassandra.db.compaction.SizeTieredCompactionStrategy', 'max_threshold': '32', 'min_threshold': '4'}
AND compression = {'chunk_length_in_kb': '64', 'class': 'org.apache.cassandra.io.compress.LZ4Compressor'}
AND crc_check_chance = 1.0
AND dclocal_read_repair_chance = 0.1
AND default_time_to_live = 0
AND gc_grace_seconds = 0
AND max_index_interval = 2048
AND memtable_flush_period_in_ms = 0
AND min_index_interval = 128
AND read_repair_chance = 0.0
AND speculative_retry = '99PERCENTILE';

查询：从test_10中选择value_1，其中year = 2016

当我运行nodetool时，它显示行缓存被命中，如下所示： 

Key Cache              : entries 144, size 103.85 KiB, capacity 100 MiB, 567 hits, 723 requests, 0.784 recent hit rate, 14400 save period in seconds
Row Cache              : entries 31, size 2.07 MiB, capacity 10 GiB, 279 hits, 310 requests, 0.900 recent hit rate, 3600 save period in seconds

但是无论是否启用行缓存，查询延迟都相同，并且在某些测试情况下，启用行缓存后延迟甚至更长。

运行跟踪时，详细信息如下：启用行缓存时： 

Tracing session: 0667b5d0-1c33-11e8-8e0d-c18210ebaedc

 activity                                                                                          | timestamp                  | source      | source_elapsed | client
---------------------------------------------------------------------------------------------------+----------------------------+-------------+----------------+-------------
                                                                                Execute CQL3 query | 2018-02-27 21:56:51.117000 | 10.81.220.6 |              0 | 10.81.220.6
 Parsing select value_0 from test_10 where year = 20160102 limit 10; [Native-Transport-Requests-1] | 2018-02-27 21:56:51.117000 | 10.81.220.6 |             87 | 10.81.220.6
                                                 Preparing statement [Native-Transport-Requests-1] | 2018-02-27 21:56:51.117000 | 10.81.220.6 |            154 | 10.81.220.6
                                                Read-repair DC_LOCAL [Native-Transport-Requests-1] | 2018-02-27 21:56:51.117000 | 10.81.220.6 |            284 | 10.81.220.6
                                                                       Row cache hit [ReadStage-3] | 2018-02-27 21:56:51.119000 | 10.81.220.6 |           2366 | 10.81.220.6
                                                  Read 10 live and 0 tombstone cells [ReadStage-3] | 2018-02-27 21:56:51.119000 | 10.81.220.6 |           2460 | 10.81.220.6
                                                                                  Request complete | 2018-02-27 21:56:51.119563 | 10.81.220.6 |           2563 | 10.81.220.6

未启用行缓存时：跟踪会话：bc0cd420-1c32-11e8-8e0d-c18210ebaedc 

 activity                                                                                          | timestamp                  | source      | source_elapsed | client
---------------------------------------------------------------------------------------------------+----------------------------+-------------+----------------+-------------
                                                                                Execute CQL3 query | 2018-02-27 21:54:46.370000 | 10.81.220.6 |              0 | 10.81.220.6
 Parsing select value_0 from test_10 where year = 20160102 limit 10; [Native-Transport-Requests-1] | 2018-02-27 21:54:46.370000 | 10.81.220.6 |            124 | 10.81.220.6
                                                 Preparing statement [Native-Transport-Requests-1] | 2018-02-27 21:54:46.370000 | 10.81.220.6 |            230 | 10.81.220.6
                                         Executing single-partition query on test_10 [ReadStage-3] | 2018-02-27 21:54:46.370000 | 10.81.220.6 |            570 | 10.81.220.6
                                                        Acquiring sstable references [ReadStage-3] | 2018-02-27 21:54:46.370000 | 10.81.220.6 |            610 | 10.81.220.6
           Skipped 0/1 non-slice-intersecting sstables, included 0 due to tombstones [ReadStage-3] | 2018-02-27 21:54:46.370000 | 10.81.220.6 |            639 | 10.81.220.6
                                                         Key cache hit for sstable 1 [ReadStage-3] | 2018-02-27 21:54:46.370001 | 10.81.220.6 |            690 | 10.81.220.6
                                           Merged data from memtables and 1 sstables [ReadStage-3] | 2018-02-27 21:54:46.371000 | 10.81.220.6 |            847 | 10.81.220.6
                                                  Read 10 live and 0 tombstone cells [ReadStage-3] | 2018-02-27 21:54:46.371000 | 10.81.220.6 |            878 | 10.81.220.6
                                                                                  Request complete | 2018-02-27 21:54:46.370998 | 10.81.220.6 |            998 | 10.81.220.6

您会看到，未启用缓存（998）显然比启用缓存（2563）快。

什么原因？？

2 个解决方案

解决方案1
 3  2018-02-26 09:46:34

您是否使用TRACING运行了一些查询，并可以发布结果？ （请参阅https://docs.datastax.com/en/cql/3.3/cql/cql_reference/cqlshTracing.html ）

当缓存命中会阻止磁盘读取时，Cassandra将受益于行缓存。 您的缓存的命中率为0.9，这已经很高。 但是由于行缓存的大小只有大约2 MB的数据，我想您之前的读取是从cassandras memtables提供的，而不是磁盘读取的，因此加速效果非常小。 

cqlsh:demo> SELECT value_1 FROM test_10 WHERE year = 2018;

 value_1
---------
     0.1
     0.1
     0.1

(3 rows)

Tracing session: 8ea1bd70-1ad9-11e8-b37d-63be3c2a5a29

 activity                                                                                        | timestamp                  | source       | source_elapsed
-------------------------------------------------------------------------------------------------+----------------------------+--------------+----------------
                                                                              Execute CQL3 query | 2018-02-26 09:43:53.799000 | 85.10.240.52 |              0
                    Parsing SELECT value_1 FROM test_10 WHERE year = 2018; [SharedPool-Worker-1] | 2018-02-26 09:43:53.800000 | 85.10.240.52 |            262
                                                       Preparing statement [SharedPool-Worker-1] | 2018-02-26 09:43:53.800000 | 85.10.240.52 |            385
                                           reading data from /85.10.240.60 [SharedPool-Worker-1] | 2018-02-26 09:43:53.800000 | 85.10.240.52 |            739
                 Sending READ message to /85.10.240.60 [MessagingService-Outgoing-/85.10.240.60] | 2018-02-26 09:43:53.800000 | 85.10.240.52 |           1005
              READ message received from /85.10.240.52 [MessagingService-Incoming-/85.10.240.52] | 2018-02-26 09:43:53.802000 | 85.10.240.60 |             19
                               Executing single-partition query on test_10 [SharedPool-Worker-1] | 2018-02-26 09:43:53.802000 | 85.10.240.60 |            184
                                              Acquiring sstable references [SharedPool-Worker-1] | 2018-02-26 09:43:53.802000 | 85.10.240.60 |            278
                                               Merging memtable tombstones [SharedPool-Worker-1] | 2018-02-26 09:43:53.802000 | 85.10.240.60 |            323
 Skipped 0/0 non-slice-intersecting sstables, included 0 due to tombstones [SharedPool-Worker-1] | 2018-02-26 09:43:53.802000 | 85.10.240.60 |            373
                                Merging data from memtables and 0 sstables [SharedPool-Worker-1] | 2018-02-26 09:43:53.802000 | 85.10.240.60 |            401
                                         Read 3 live and 0 tombstone cells [SharedPool-Worker-1] | 2018-02-26 09:43:53.802001 | 85.10.240.60 |            608
                                       Enqueuing response to /85.10.240.52 [SharedPool-Worker-1] | 2018-02-26 09:43:53.803000 | 85.10.240.60 |            709
  REQUEST_RESPONSE message received from /85.10.240.60 [MessagingService-Incoming-/85.10.240.60] | 2018-02-26 09:43:53.803000 | 85.10.240.52 |           3972
     Sending REQUEST_RESPONSE message to /85.10.240.52 [MessagingService-Outgoing-/85.10.240.52] | 2018-02-26 09:43:53.803000 | 85.10.240.60 |            948
                                    Processing response from /85.10.240.60 [SharedPool-Worker-2] | 2018-02-26 09:43:53.804000 | 85.10.240.52 |           4060
                                                                                Request complete | 2018-02-26 09:43:53.803553 | 85.10.240.52 |           4553


cqlsh:demo>

相关部分是： 

Skipped 0/0 non-slice-intersecting sstables, included 0 due to tombstones [SharedPool-Worker-1] | 2018-02-26 09:43:53.802000 | 85.10.240.60 |            373
                               Merging data from memtables and 0 sstables [SharedPool-Worker-1] | 2018-02-26 09:43:53.802000 | 85.10.240.60 |            401

在我的小型演示案例中，没有任何稳定的读取或写入操作。

解决方案2
 0  2018-02-26 14:09:05

如果关闭行缓存，则似乎读取sstable。 当行缓存打开时，跟踪如下：

从test_10中选择value_1，其中year = 20160102 

    Tracing session: 71705c70-1afc-11e8-892f-edfe3d6c5915

     activity                                                                                 | timestamp                  | source      | source_elapsed | client
    ------------------------------------------------------------------------------------------+----------------------------+-------------+----------------+-------------
                                                                           Execute CQL3 query | 2018-02-26 08:53:37.208000 | 10.81.220.6 |              0 | 10.81.220.6
     Parsing select value_1 from test_10 where year = 20160102; [Native-Transport-Requests-1] | 2018-02-26 08:53:37.208000 | 10.81.220.6 |            318 | 10.81.220.6
                                            Preparing statement [Native-Transport-Requests-1] | 2018-02-26 08:53:37.208000 | 10.81.220.6 |            762 | 10.81.220.6
                                                                  Row cache hit [ReadStage-2] | 2018-02-26 08:53:37.211000 | 10.81.220.6 |           3433 | 10.81.220.6
                                            Read 100 live and 0 tombstone cells [ReadStage-2] | 2018-02-26 08:53:37.211000 | 10.81.220.6 |           3955 | 10.81.220.6
                                                                             Request complete | 2018-02-26 08:53:37.223503 | 10.81.220.6 |          15503 | 10.81.220.6



    Tracing session: 722ad2d0-1afc-11e8-892f-edfe3d6c5915

     activity                                                                                 | timestamp                  | source      | source_elapsed | client
    ------------------------------------------------------------------------------------------+----------------------------+-------------+----------------+-------------
                                                                           Execute CQL3 query | 2018-02-26 08:53:38.429000 | 10.81.220.6 |              0 | 10.81.220.6
     Parsing select value_1 from test_10 where year = 20160102; [Native-Transport-Requests-1] | 2018-02-26 08:53:38.429000 | 10.81.220.6 |            381 | 10.81.220.6
                                            Preparing statement [Native-Transport-Requests-1] | 2018-02-26 08:53:38.429000 | 10.81.220.6 |            592 | 10.81.220.6
                                                                  Row cache hit [ReadStage-2] | 2018-02-26 08:53:38.437000 | 10.81.220.6 |           8411 | 10.81.220.6
                                            Read 100 live and 0 tombstone cells [ReadStage-2] | 2018-02-26 08:53:38.438000 | 10.81.220.6 |           8720 | 10.81.220.6
                                                                             Request complete | 2018-02-26 08:53:38.449057 | 10.81.220.6 |          20057 | 10.81.220.6

当缓存关闭时： 

    Tracing session: efaad070-1afc-11e8-bafb-efaf5c6d9339

     activity                                                                                 | timestamp                  | source      | source_elapsed | client
    ------------------------------------------------------------------------------------------+----------------------------+-------------+----------------+-------------
                                                                           Execute CQL3 query | 2018-02-26 08:57:08.984000 | 10.81.220.6 |              0 | 10.81.220.6
     Parsing select value_1 from test_10 where year = 20160102; [Native-Transport-Requests-1] | 2018-02-26 08:57:08.988000 | 10.81.220.6 |           4448 | 10.81.220.6
                                            Preparing statement [Native-Transport-Requests-1] | 2018-02-26 08:57:08.989000 | 10.81.220.6 |           4730 | 10.81.220.6
                                    Executing single-partition query on test_10 [ReadStage-3] | 2018-02-26 08:57:08.993000 | 10.81.220.6 |           9314 | 10.81.220.6
                                                   Acquiring sstable references [ReadStage-3] | 2018-02-26 08:57:08.993000 | 10.81.220.6 |           9493 | 10.81.220.6
      Skipped 0/1 non-slice-intersecting sstables, included 0 due to tombstones [ReadStage-3] | 2018-02-26 08:57:08.994000 | 10.81.220.6 |           9727 | 10.81.220.6
                             Partition index with 2 entries found for sstable 1 [ReadStage-3] | 2018-02-26 08:57:08.994000 | 10.81.220.6 |          10200 | 10.81.220.6
                                      Merged data from memtables and 1 sstables [ReadStage-3] | 2018-02-26 08:57:08.997000 | 10.81.220.6 |          13002 | 10.81.220.6
                                            Read 100 live and 0 tombstone cells [ReadStage-3] | 2018-02-26 08:57:08.997000 | 10.81.220.6 |          13265 | 10.81.220.6
                                                                             Request complete | 2018-02-26 08:57:08.998950 | 10.81.220.6 |          14950 | 10.81.220.6



    Tracing session: f0ad9a70-1afc-11e8-bafb-efaf5c6d9339

     activity                                                                                 | timestamp                  | source      | source_elapsed | client
    ------------------------------------------------------------------------------------------+----------------------------+-------------+----------------+-------------
                                                                           Execute CQL3 query | 2018-02-26 08:57:10.679000 | 10.81.220.6 |              0 | 10.81.220.6
     Parsing select value_1 from test_10 where year = 20160102; [Native-Transport-Requests-1] | 2018-02-26 08:57:10.679000 | 10.81.220.6 |            355 | 10.81.220.6
                                            Preparing statement [Native-Transport-Requests-1] | 2018-02-26 08:57:10.680000 | 10.81.220.6 |            752 | 10.81.220.6
                                    Executing single-partition query on test_10 [ReadStage-2] | 2018-02-26 08:57:10.682000 | 10.81.220.6 |           3110 | 10.81.220.6
                                                   Acquiring sstable references [ReadStage-2] | 2018-02-26 08:57:10.682000 | 10.81.220.6 |           3339 | 10.81.220.6
      Skipped 0/1 non-slice-intersecting sstables, included 0 due to tombstones [ReadStage-2] | 2018-02-26 08:57:10.682000 | 10.81.220.6 |           3477 | 10.81.220.6
                                                    Key cache hit for sstable 1 [ReadStage-2] | 2018-02-26 08:57:10.683000 | 10.81.220.6 |           3598 | 10.81.220.6
                                      Merged data from memtables and 1 sstables [ReadStage-2] | 2018-02-26 08:57:10.684000 | 10.81.220.6 |           5438 | 10.81.220.6
                                            Read 100 live and 0 tombstone cells [ReadStage-2] | 2018-02-26 08:57:10.685000 | 10.81.220.6 |           5593 | 10.81.220.6
                                                                             Request complete | 2018-02-26 08:57:10.685204 | 10.81.220.6 |           6204 | 10.81.220.6

暂无
暂无

声明:本站的技术帖子网页，遵循CC BY-SA 4.0协议，如果您需要转载，请注明本站网址或者原文地址。任何问题请咨询:yoyou2525@163.com.

相关问题 在 postgreSQL 中启用查询缓存以提高性能 cassandra和行缓存更新 缓存资源文件以提高性能 缓存对齐可以提高性能吗？ 如何在此MIPS代码上提高缓存性能 缓存如何提高流水线处理器的性能？ 如何缓存我的JavaScript文件以提高性能 用Redis来休眠二级缓存 - 它会提高性能吗？ 哪种类型的缓存更适合用来提高网站的性能？ 通过使用Redis作为缓存，可以提高HBase的读取性能吗？
相关标签
 
粤ICP备18138465号  © 2020-2024 STACKOOM.COM