MySQL查询语句索引调优

Question

I'm working on how to implement a global leaderboard for a Facebook racing game my company has released. What I'd like to do is be able to store the player's userID and their time for a race. I've got a table like the one below:

+--------+-----------------------+------+-----+---------+-------+
| Field  | Type                  | Null | Key | Default | Extra |
+--------+-----------------------+------+-----+---------+-------+
| userID | mediumint(8) unsigned | NO   | PRI | 0       |       |
| time   | time                  | YES  | MUL | NULL    |       |
+--------+-----------------------+------+-----+---------+-------+

And a sample set of data like so:

+--------+----------+
| userID | time     |
+--------+----------+
| 505610 | 10:10:10 |
| 544222 | 10:10:10 |
| 547278 | 10:10:10 |
| 659241 | 10:10:10 |
| 681087 | 10:10:10 |
+--------+----------+

My queries will be coming from PHP. Now if I assume that I've got unlimited resources, what I could do is this:

$q1 = "Set @rank := 0";
$q2 = "select @rank:=@rank+1 as rank,userID,time from highscore order by time asc where userID=$someUserID";
$q3 = "Set @rank := 0";
$q4 = "select @rank:=@rank+1 as rank,userID,time from highscore order by time asc where rank > $rankFromSecondQuery - 10 and rank < $rankFromSecondQuery + 10";

But I don't have unlimited resources and I have to be able to scale this to support millions of players since it's going into a social game on Facebook. So after spending a few days crawling all over Google, I've been able to get my queries down to this:

$q5 = "select rank,userID,time from (select @rank:=0) r, (select @rank:=@rank+1 as rank,userID,time from highscore order by time asc) as myMine where userID=$someUserID"
$q6 = "select rank,userID,time from (select @rank:=0) r, (select @rank:=@rank+1 as rank,userID,time from highscore order by time asc) as myMine where rank > $rankFromFirstQuery - 10 and rank < $rankFromSecondQuery + 10";

This works, but it isn't very pretty with an average runtime per query of approx 2.3 seconds.

EDIT: Here's what the $q5 and $q6 give me when run them:

mysql> select rank,userID,time from (select @rank:=0) r, (select @rank:=@rank+1 as rank,userID,time from highscore order by time asc) as myMine where userID=11345;                                                                          
+--------+--------+----------+
| rank   | userID | time     |
+--------+--------+----------+
| 423105 |  11345 | 12:47:23 |
+--------+--------+----------+
1 row in set (2.42 sec)

mysql> select rank,userID,time from (select @rank:=0) r, (select @rank:=@rank+1 as rank,userID,time from highscore order by time asc) as myMine where rank>423100 and rank<423110;
+--------+---------+----------+
| rank   | userID  | time     |
+--------+---------+----------+
| 423101 | 2416665 | 12:47:22 |
| 423102 | 2419720 | 12:47:22 |
| 423103 | 2426606 | 12:47:22 |
| 423104 | 2488517 | 12:47:22 |
| 423105 |   11345 | 12:47:23 |
| 423106 |   92350 | 12:47:23 |
| 423107 |   94277 | 12:47:23 |
| 423108 |  114685 | 12:47:23 |
| 423109 |  135434 | 12:47:23 |
+--------+---------+----------+
9 rows in set (2.58 sec)

Here's the explain extended block $q5 and the one for $q6 looks just about identical:

mysql> explain select rank,userID,time from (select @rank:=0) r, (select @rank:=@rank+1 as rank,userID,time from highscore order by time asc) as myMine where userID=11345;
+----+-------------+------------+--------+---------------+----------+---------+------+---------+----------------+
| id | select_type | table      | type   | possible_keys | key      | key_len | ref  | rows    | Extra          |
+----+-------------+------------+--------+---------------+----------+---------+------+---------+----------------+
|  1 | PRIMARY     | <derived2> | system | NULL          | NULL     | NULL    | NULL |       1 |                |
|  1 | PRIMARY     | <derived3> | ALL    | NULL          | NULL     | NULL    | NULL | 2500000 | Using where    |
|  3 | DERIVED     | highscore  | index  | NULL          | idx_time | 4       | NULL | 2500842 | Using index    |
|  2 | DERIVED     | NULL       | NULL   | NULL          | NULL     | NULL    | NULL |    NULL | No tables used |
+----+-------------+------------+--------+---------------+----------+---------+------+---------+----------------+

So ultimately, what I'd really like to be able to do is get this down into just one query such that I can temper the execution time with a high CPU server or two. Either that or I'd like to figure out a way to just hit an index on the part of the query that's associated with the derive3 line in the explain block that's hitting on all the rows in the table.

Here's a couple of the queries that I've tried without any success so far:

select rank,userID,time from (select @rank:=0) r, (select @playerRank := rank from (select @rank:=@rank+1 as rank,userID,time from highscore order by time asc) as myMine where userID=11345) as myFoo where @playerRank>423100 and @playerRank<423110;
select rank,userID,time from (select @playerRank := rank from (select @rank := 0) r, (select @rank:=@rank+1 as rank,userID,time from highscore order by time asc) as myMine where userID=11345) as myFoo where @playerRank>423100 and @playerRank<423110;
select * from (select @rank:=0) r, (select @playerRank := userID from (select @rank:=@rank+1 as rank,userID,time from highscore order by time asc) as myMine where userID=11345) as myFoo where @playerRank>423100 and @playerRank<423110;

The first two game me a "ERROR 1054 (42S22): Unknown colum 'rank' in 'field list' error and the third just returned an empty set instead of the data I was looking for.

Anyone have any ideas how to either get my two queries listed above to hit on an index so the execution time decreases or how to combine the two queries into one so I only have to suffer with a painful execution time once? I'd also be open to tuning/optimizations like tweaking MySQL config settings and/or using something like Percona if anyone has experience using something like that and would like to share their experiences.

Answer 1

After runnint $q5 you shold know the rank of the user, after that you should be able to use a limit to get the right rows

$lowest_rank_to_fetch = max(0, $rankFromFirstQuery - 10);
$q6l = "SELECT userID, time
        FROM highscore
        ORDER BY time ASC
        LIMIT {$lowest_rank_to_fetch}, 21";

/* some execute query function */

foreach(range($lowest_rank_to_fetch, $lowest_rank_to_fetch+21) as $current_rank)
{
   /* some database fetch function */
   /* add $current_rank to result */
}

Answer 2

You can obtain the rank first using count() this should perform a bit better for the first query:

SELECT COUNT(h.userID) as rank, h2.userID, h2.time
   FROM highscore h
   LEFT OUTER JOIN highscore h2 ON (h.time <= h2.time)
   WHERE h2.userID = ?

Then you could use Puggan's technique for querying the nearby rankings.

SELECT ... ORDER BY time LIMIT $lowest_rank, 21

Answer 3

I'd like to propose this alternative solution to get at what you're trying to achieve.

Make a separate table to store the rank. Don't compute it every time a user wants to know his/her rank, and don't include it in the existing table. Putting the ranks in a separate table will hopefully ease lock contention problems when score updates compete with the rank computation.

Recalculate the ranks at a regular interval. When you do this recalculation, do it by truncating the ranks table and recreating it from scratch. Doing this with either a bulk load operation (LOAD DATA INFILE) or make it a MyISAM table (which is fast when inserting at the end of the table). Either way should be relatively fast to actually write out the table; faster, at least, than updating millions of rows in a table that's already in place. Both of these methods make your ranks table brittle and vulnerable to loss in the event of a crash, but that's OK because this is essentially transient data. As long as your scores table is stable, you're safe. By recalculating at regular intervals, you avoid the problem of having to do the calculation more and more often as the number of plays increases until you hit a wall.

If the user scores within the top 100, push out their new score right away. Users may want to browse the top 100 to see who has the highest scores. I see little likelihood of anyone wanting to actually browse the list below that point.

Allow users to see their friends' scores immediately, along with their relative rank compared to each other. This is probably the ranking that most users are interested in. I know when my wife plays a Facebook game, she has no interest in her overall ranking but she very much wants to know if she beat her college classmates.

Show the player's overall rank, and those of their friends, as invalidated after the user's latest play, and load them asynchronously whenever the next update is ready.

Another consideration is that, if this game will be around for a few years, your scoreboard will end up clogged with old scores from inactive players, especially around the low end. You may want to consider whether it's worthwhile to archive off these scores. For example, you could say that any player in the lower 75% of the scoreboard will only be considered in the ranking if they played within the last 6 months. Then, move their scores out to an archive table, where they will be remembered and can be restored to the scoreboard if that player returns but won't have to be included in the sort every time you calculate the ranking. Yes, this will arguably make your ranking less "true", but people are just playing for fun anyway. It would have the side effect of making their rankings look better which is also fun. Some fine print on the scoreboard will briefly mention that old scores aren't included, so you can still say everything is above the board.