Display 10 Random Rows from MySQL Table w/ PHP

Question

I've been doing tons of research on this but can't find an answer which is right for my situation.

I run a website where details of around 10,000 products are stored in a MySQL database. This number is always increasing and eventually could get up to 20,30 or 40,000 products.

I'm in the middle of a redesign of the site, and as part of this I want to show 10 random products from the table that the stores the products. Ideally I want to add pagination so you if you wanted, you could scroll through all 10,000 products - but the pagination isn't the issue.

I can't find an efficient way to grab random rows from the MySQL table. I've seen a lot of mention of RAND() but it's super slow with anything over a 100 or so rows. I've tried things like generating a random number based on the number of entries and pulling by that - but that doesn't account for if a row is deleted and then it wouldn't find a result. It also means I would have to run the statement 10 times for 10 random rows.

I've thought about storing all rows in to an array and then pulling 10 random rows. But it also seems inefficient to store a min of 10,000 rows into an array every time a user visits this page.

Does anyone have any advice on the best way to do this? I know I haven't posted any actual code here, but that's because I don't have any. Every method I've tried doesn't do what I need or is super slow.

Any advice would be appreciated:(

EDIT; The SQL query I'm running that takes 18 seconds with Rand() is:

SELECT Title,Price,Img,Seller,Positive,Negative,Neutral,Glowing,PD,STOCK,StoreItems.ItemURL,Currency,Verified, BTC, ETH, LTC, PayPal, Stripe, Stores.Verified FROM Stores JOIN StoreItems On Stores.StoreID = StoreItems.StoreID INNER JOIN Promotions ON StoreItems.ItemURL = Promotions.ItemID INNER JOIN Gateway ON StoreItems.ItemURL = Gateway.ItemURL WHERE Stores.Verified = 'true' ORDER BY RAND() LIMIT 5

Answer 1

You could generate 10 pseudo random values with PHP rand and try to match the ID's

$r = [];
for ($i = 0; $i<10;$i++) {$r[] = rand(0, NUM_OF_PRODUCTS);}
$stmt = $conn->prepare("SELECT 1 FROM products WHERE id IN (?,?,?,?,?,?,?,?,?,?)");
$stmt->bind_param('iiiiiiiiii', $r[], $r[1], $r[2], $r[3], $r[4], $r[5], $r[6], $r[7], $r[8], $r[9]);

Answer 2

Define super slow.

I just pulled 50 random rows from a table of 304,096 rows with an execution time of 0.123327 seconds with the following query:

SELECT *
FROM `calendar`
ORDER BY RAND()
LIMIT 50;

Performance measured with this query:

SET profiling = 1;
SELECT *
FROM `calendar`
ORDER BY RAND()
LIMIT 50;
SELECT query_id, SUM(duration)
FROM information_schema.profiling
GROUP BY query_id
ORDER BY query_id
DESC LIMIT 3;

Answer 3

I would assign them random numbers with a cron, either daily or hourly, like 1 to 10,000, with one UPDATE query.

UPDATE products SET static_random = FLOOR(RAND()*10000)

So instead of getting a random assignment each time, the "randomness" is done on a schedule. (Or just use the default RAND() which does between 0 and 1) Then on the page, pick a random number with PHP ($randomPosition}) within the same range and then return from MySQL something like:

SELECT * FROM products WHERE static_random > {$randomPosition} 
ORDER BY static_random ASC LIMIT 10

So it would pick the next 10 "random" items after the position randomly picked. If in the small chance that there is less then 10 returns, then pick another random position.

Depending on how often you run the cron, it may not be completely random, but it would seem random enough, because of your chances of picking the same range of 10 numbers out of 10,000 would be small enough I would think.

There are others way to randomize the assigned static random number instead of a cron. Maybe randomly 1 out 100 chance on a page visit or so, assign them all random numbers again in one UPDATE query (and definitely not individual UPDATE queries). Or out of the 10 that were just picked, generate a new static random numbers for those, so a small batch gets "re-shuffled" back into the deck at random spots each time.

Or instead you can alway pick the first 10 items ordered by their static random number, and not used a random position. Always start with the first position. This may help with pagination to get the same "random" result sets each time.

In addition, for those items selected, you can generate new random numbers, possibly on the higher range end. That way, they go toward the "end of the stack" and are less likely to be selected again until other items gets a turn. Still, at some point you may have to reshuffle them all.