mysql order by query issue

Question

I'm having some problem in using order by in mysql. I have a table called "site" with 3 fields like id,name,rank. This table consists around 1.4m records. when i apply query like,

select name from site limit 50000,10;  

it returns 10 records in 7.45 seconds [checked via terminal]. But when i use order by in the above query like,

select name from site order by id limit 50000,10;  

the query never seems to be complete. Since the id is set as primary key, i thought it doesn't need another indexing to speedup my query. but i don't know where is the mistake.

Any help greatly appreciated, Thanks.

Answer 1

This is "to be expected" with large LIMIT values:

From http://www.mysqlperformanceblog.com/2006/09/01/order-by-limit-performance-optimization/

Beware of large LIMIT Using index to sort is efficient if you need first few rows, even if some extra filtering takes place so you need to scan more rows by index then requested by LIMIT. However if you're dealing with LIMIT query with large offset efficiency will suffer. LIMIT 1000,10 is likely to be way slower than LIMIT 0,10. It is true most users will not go further than 10 page in results, however Search Engine Bots may very well do so. I've seen bots looking at 200+ page in my projects. Also for many web sites failing to take care of this provides very easy task to launch a DOS attack – request page with some large number from few connections and it is enough. If you do not do anything else make sure you block requests with too large page numbers.

For some cases, for example if results are static it may make sense to precompute results so you can query them for positions. So instead of query with LIMIT 1000,10 you will have WHERE position between 1000 and 1009 which has same efficiency for any position (as long as it is indexed)

AND

One more note about ORDER BY … LIMIT is – it provides scary explain statements and may end up in slow query log as query which does not use indexes

The last point is THE important point in your case - the combination of ORDER BY and LIMIT with a big table (1.4m) and the "not use indexes" (even if there are indexes!) in this case makes for really slow performance...

EDIT - as per comment:

For this specific case you should use select name from site order by id and handle the splitting of the resultset into chunks of 50,000 each in your code!

Answer 2

Can you try this:

SELECT name 
FROM site 
WHERE id >= ( SELECT id
              FROM site
              ORDER BY id
              LIMIT 50000, 1
            )
ORDER BY id
LIMIT 10 ;