Rails 3.1-Heroku上的MySQL和PostgreSQL之间的巨大查询时间差

Question

I have a query in my dev environment that typically takes about 1.7 ms to run on the dev mySQL database. When bumped up to Heroku and PostgreSQL, the same query on the same data set is taking about 1.2 seconds !

SELECT distinct user_id, score, quality 
FROM `reports` 
WHERE (datetime_utc >= '2012-01-13 14:00:00' AND 
       datetime_utc <= '2012-01-14 14:00:00') 
ORDER BY score DESC, quality DESC LIMIT 20

I created a compound index on score and quality which helped with the SQL version, but the query running on PostgreSQL is still very, very slow. My first instinct is to check that the index is actually in place on the Heroku side, but i'm not sure quite how to do that - in any case, i have a feeling it's more to do with the fact that mySQL and PostgreSQL don't do things quite the same way.

Any insights or pointers would be hugely appreciated!

Answer 1

Try this modified query:

SELECT user_id, score, quality
FROM   reports
WHERE  datetime_utc BETWEEN '2012-01-13 14:00:00' AND '2012-01-14 14:00:00'
GROUP  BY user_id, score, quality
ORDER  BY score DESC, quality DESC
LIMIT  20

• Since DISTINCT is applied last, it may be slower than GROUP BY with many non-distinct rows. You'd have to test - with EXPLAIN ANALYZE . Otherwise, the result is the same.

• Minor simplification to the WHERE clause with BETWEEN . Removed non-standard MySQL syntax.

• An Index on (score, quality) will hardly get used. The useful index here is (should make a big difference in most scenarios):

CREATE INDEX reports_date_time_utc_idx ON reports (date_time_utc)

The important part is the index.

Answer 2

Could the difference in performance be caused by size of datasets between dev and heroku?

Having an index on score,quality will not help much if there are many rows, since it still must filter based on datetime_utc.

You may wish to consider an index on datetime_utc, since it needs to filter there first.

If you really want to optimize for read speed, you could have a compound index on datetime_utc, score, quality, user_id which would completely eliminate the need to lookup the row data.

However, beware of doing that, since you may then cause a hotspot on inserts with such a wide index.

Answer 3

As Heroku doesn't allow you to connect to the database unless having one of the >200$/month plan you could try to retrieve a local copy of the database for local inspection.

heroku db:pull // Will give you a local copy of the db

The result will be something like this:

Receiving schema
Receiving data
8 tables, 591 records
users:         100% |================================| Time: 00:00:00
pages:         100% |================================| Time: 00:00:00
comments:      100% |================================| Time: 00:00:00
tags:          100% |================================| Time: 00:00:00
Receiving indexes
Resetting sequences