Optimizing SQL Query, large table - query kills server

Question

I have this script to get Posts:

$totalrows = 60;

$sql = "SELECT 
 posts.Tags as tags, 
 posts.OwnerUserId as postsid, 
 posts.Id as postid, 
 posts.Body as body, 
 posts.Title as title, 
 users.Id as userid, 
 users.DisplayName as usersname  
FROM posts 
JOIN users ON posts.OwnerUserId = users.Id 
JOIN (select posts.id from posts where posts.title != '' order by rand() asc limit " . $totalrows .") AS tmp_result
ON (posts.Id = tmp_result.Id)";


$r = mysql_query($sql) or die(mysql_error());

The problem is the server is freezing and require restarting, the mysql file is very large. What causing freezing the server? What can I do to optimize the above query?

Answer 1

Order by rand() is really expensive, so you might want to consider doing something like pickint the id's beforehand in code, and asking for those particular ones.

Furthermore, use an "EXPLAIN" on the query, and see what happens. If i'm not reading this wrong, you'll probably see the subquery in there, and you can't use an index on that (because it's a 'new' table, so it doesn't have an index.

As a last point, check the index(es?) of your tables.

Answer 2

A simplification - your self-join on posts could be

JOIN posts AS p2 ON posts.id = p2.id AND p2.title != '' (should that be p2.title IS NOT NULL?)

Answer 3

ORDER BY RAND() does not scale.

The answer to mysql-alternatives-to-order-by-rand provides a smart way of returning rows in random order.

Also, try adding LIMIT 100 at the end of your query to see if it helps.