mysql解释类型索引与所有性能问题

Question

I have the following table below: I have 3.5 million records.

CREATE TABLE `video_downloads` (
  `id` bigint(20) NOT NULL AUTO_INCREMENT,
  `user_id` bigint(20) NOT NULL,
  `video_id` bigint(20) NOT NULL,
  `download_at` datetime NOT NULL,
  PRIMARY KEY (`id`)
) ENGINE=InnoDB AUTO_INCREMENT=3573041 DEFAULT CHARSET=latin1

Only id and user_id are indexed .

Here is my query:

select max(video_id), user_id
from video_downloads
group by user_id

With this current table setup, this query took me about more than 10 mins to run. So here is the explain

| id | select_type | table           | type  | possible_keys | key     | key_len | ref | rows    | Extra |
|----|-------------|-----------------|-------|---------------|---------|---------|-----|---------|-------|
| 1  | SIMPLE      | video_downloads | index |               | user_id | 8       |     | 3562709 |       |

{
  "query_block": {
    "select_id": 1,
    "table": {
      "table_name": "video_downloads",
      "access_type": "index",
      "key": "user_id",
      "key_length": "8",
      "used_key_parts": ["user_id"],
      "rows": 3562709,
      "filtered": 100
    }
  }
}

And then I removed the index for user_id running the same query and it took me about 1.5 s .

Here is the explain without user_id index

| id | select_type | table           | type | possible_keys | key | key_len | ref | rows    | Extra                           |
|----|-------------|-----------------|------|---------------|-----|---------|-----|---------|---------------------------------|
| 1  | SIMPLE      | video_downloads | ALL  |               |     |         |     | 3562709 | Using temporary; Using filesort |

{
  "query_block": {
    "select_id": 1,
    "filesort": {
      "sort_key": "video_downloads.user_id",
      "temporary_table": {
        "table": {
          "table_name": "video_downloads",
          "access_type": "ALL",
          "rows": 3562709,
          "filtered": 100
        }
      }
    }
  }
}

I think my main question is why there is such a huge difference in terms of the time with and without the index on user_id . And when there is an index on user_id the type is index which means it's using the index but the query is very slow.

I am a bit confused with the result I do not think I understand why this is happening, I check the official doc still not fully understand it.

update I think the main reason could be it's using the index data to fetch the row from disk, and it's going to do it one by one and randomly. So that's 3.5 million times random read from disk. That's the only reason I can think of. However, is that going to be that slow? (more than 10 mins vs 1.5 s ???).

However, from MySQL doc

Sometimes MySQL does not use an index, even if one is available. One circumstance under which this occurs is when the optimizer estimates that using the index would require MySQL to access a very large percentage of the rows in the table.

In my case, MySQL does not seem to take the right decision. I can see the possible_keys is null but key is still using the index why? is it because of the group by ?

Answer 1

The "statistics" that the Optimizer uses are not always perfect. However "10 min" vs "1.5 sec" is quite spectacular. I wonder if there was outside interference. Oh, what Engine is being used?

When it used the single-column index, it probably had to bounce between the index and the data, fetching 3.5M rows one at a time, but randomly.

When it did the table scan ("All"), it also read 3.5M rows, but sequentially. But then it had to followup with a sort.

Buffer_pool

16M for innodb_buffer_pool_size is the problem. Set that to about 70% of RAM size unless you have an especially small machine.

The 10-minute query was probably solid I/O, reading and rereading the data from the table randomly .

On a spinning disk (HDD, not SDD), 3.5M reads at 100 blocks/second is several hours. So you were lucky to get finished in only 10 minutes. The 1.5s says how useful a big enough RAM cache is.

The 1.5s may be all it took to read straight (not randomly) through the entire table once.