将所有分区用于MySQL中基本SELECT语句的分区表

Question

I have a table in MySQL partitioned by HASH on the function of year(date) . The goal is to distribute my data into a partition for each year more or less.

When executing a basic select statement:

EXPLAIN PARTITIONS
SELECT date 
FROM date_table 
WHERE date >= '2008-01-01' AND date <= '2009-01-01'

...all partitions are being used. I would assume that only some of the partitions would be used, at max, 2. What am I missing here regarding how partitions work?

test.sql

DROP TABLE IF EXISTS `tmp_date_table`;

CREATE TABLE `tmp_date_table` (
    `date_id` INT(11) NOT NULL,
    `date` DATE NOT NULL,
    PRIMARY KEY (`date_id`, `date`)
)
COLLATE='utf8_general_ci'
ENGINE=InnoDB
PARTITION BY HASH (year(date))
PARTITIONS 11 
;

INSERT INTO `tmp_date_table`(date_id, date) 
VALUES
(1, '2000-01-01'),
(2, '2001-01-01'),
(3, '2002-01-01'),
(4, '2003-01-01'),
(5, '2004-01-01'),
(6, '2005-01-01'),
(7, '2006-01-01'),
(8, '2007-01-01'),
(9, '2008-01-01'),
(10, '2009-01-01'),
(11, '2010-01-01');

EXPLAIN PARTITIONS
SELECT date FROM tmp_date_table WHERE date >= '2008-01-01' AND date <= '2009-01-01';

DROP TABLE IF EXISTS `tmp_date_table`;

Any help is appreciated.

Answer 1

So it looks like you are setting up correctly, I digged a little deeper.

http://dev.mysql.com/doc/refman/5.7/en/partitioning-pruning.html

When a table is partitioned by HASH or [LINEAR] KEY, pruning can be used only on integer columns. For example, this statement cannot use pruning because dob is a DATE column:

SELECT * FROM t4 WHERE dob >= '2001-04-14' AND dob <= '2005-10-15';

So you can't do what you are doing with HASH .

However, if the table stores year values in an INT column, then a query having WHERE year_col >= 2001 AND year_col <= 2005 can be pruned.

That seems counter intuitive to me, but part of the deal is that you have to always have to specify the amount of partitions up front (in your case, 11), so the partition is calculated thusly:

If you insert a record into t1 whose col3 value is '2005-09-15', then the partition in which it is stored is determined as follows:

MOD(YEAR('2010-09-01'),11)
=  MOD(2010,11)
=  8

So that will go into partition 8 rather than partition 11, which means:

MOD(YEAR('2000-09-01'),11)
=  MOD(2000,11)
=  9

Your first year would go into partition 9. It would use the correct partition if you queried on the date alone:

WHERE date = "2010-01-01"

But not on a range.

Since the range of your data is known, and it all looks historical, you will have to bite the bullet and set up a range for each year. This way however, your range query will use only the correct partitions when you use a BETWEEN .

DROP TABLE IF EXISTS `tmp_date_table`;

CREATE TABLE `tmp_date_table` (
    `date_id` INT(11) NOT NULL,
    `dates` DATE NOT NULL
)
COLLATE='utf8_general_ci'
ENGINE=InnoDB
PARTITION BY RANGE ( YEAR(`dates`) ) (
    PARTITION p0 VALUES LESS THAN (2001),
    PARTITION p1 VALUES LESS THAN (2002),
    PARTITION p2 VALUES LESS THAN (2003),
    PARTITION p3 VALUES LESS THAN (2004),
    PARTITION p4 VALUES LESS THAN (2005),
    PARTITION p5 VALUES LESS THAN (2006),
    PARTITION p6 VALUES LESS THAN (2007),
    PARTITION p7 VALUES LESS THAN (2009),
    PARTITION p8 VALUES LESS THAN (2010),
    PARTITION p9 VALUES LESS THAN (2011),
    PARTITION p10 VALUES LESS THAN MAXVALUE

);

INSERT INTO `tmp_date_table`(date_id, `dates`) 
VALUES
(1, '2000-01-01'),
(2, '2001-01-01'),
(3, '2002-01-01'),
(4, '2003-01-01'),
(5, '2004-01-01'),
(6, '2005-01-01'),
(7, '2006-01-01'),
(8, '2007-01-01'),
(9, '2008-01-01'),
(10, '2009-01-01'),
(11, '2010-01-01'),
(12, '2012-01-01');



EXPLAIN PARTITIONS
SELECT dates FROM tmp_date_table WHERE (`dates`) BETWEEN  "2001-01-01" and "2004-01-01" ;

DROP TABLE IF EXISTS `tmp_date_table`;

Answer 2

You have found a major reason why PARTITION BY HASH is virtually useless.

But, more basic... WHY do this?

CREATE TABLE `tmp_date_table` (
    `date_id` INT(11) NOT NULL,
    `date` DATE NOT NULL,
    PRIMARY KEY (`date_id`, `date`)
)

Are you trying to 'normalize' dates to date_id's?

1. date_id is INT which occupies 4 bytes. DATE occupies only 3 bytes. So this normalization wastes space.

2. Don't normalize "continuous" things such as number, dates, floats, etc. It prevents you from efficiently looking up "ranges" of such values.