对于数据库查询，返回与您所关注的人的Twitter推文源类似的结果的最佳查询方法是什么？

Question

My website lets users submit posts and subscribe to posts by other people. The homepage of the site displays the most recent posts by the people the user follows. There is no limit to the number of the people a user can follow. Some users are following thousands of other users. Some users have made more than 15,000 posts.

The posts database table is is organized like this (a few irrelevant columns are omitted for clarity):

id
author_id
post_content
date_added

I have 2 working solutions, but I'm not sure if either is the best approach:

Solution 1:

1. Get the list of author_ids a user is following.

2. Query the table for posts that match any of the author_ids:

     SELECT id FROM posts WHERE author_id IN (12, 34, 56, 78, 90, ...) ORDER BY date_time DESC LIMIT 100; 

3. Cache the result for N minutes.

This works, but crawls when users are following thousands of people.

Solution 2:

1. Get the list of author_ids a user is following.
2. For each author id, get the cached feed of just their post ids. (This feed is used on an author's page)
3. Merge all the post ids from all of these authors into one giant array and sort them in descending order (which happens to work because each post gets an auto-incremented id).
4. Cache and return the most recent 100 post ids;

This works, but sometimes crawls when thousands of user feeds are returned and merged into an array with 100,000+ items. It feels like overkill when all I care about is the most recent 100 items. Additionally, not all user feeds will be in cache. Some old users may no longer use the site, but are still followed by new users resulting in the old user's feed to be freshly queried (and then cached).

Are these the optimal solutions? If not, what is?

Answer 1

What about (untested, but you get the idea):

SELECT id FROM posts
CROSS JOIN followers ON posts.author_id = followers.user_id
WHERE followers.followed_by_user_id = INSERT_USER_ID_HERE
ORDER BY posts.date_time DESC
LIMIT 100;

or

SELECT id FROM posts
WHERE author_id IN (
  SELECT user_id FROM followers 
  WHERE followed_by_user_id = INSERT_USER_ID_HERE
)
ORDER BY date_time DESC
LIMIT 100;

note: to clarify, the table followers contains two columns user_id and followed_by_user_id . If a row contains the value ( user_id:7 , followed_by_user_id:42 ), it means that user 42 follows user 7.

Answer 2

An optimization for your Solution 2 which avoids merging and sorting all the post ids:

1. Create an array to hold the result and copy the contents of the first author's top-100 post ids and sort by id .
2. For each author:
   1. Check if the minimum id in the result array is greater than the maximum id of the author's posts.
   2. If yes, then skip that author since all his posts are older than the posts in your result array.
   3. If no, then merge the top-100 posts of the author with your result array, sort and then retain only the top 100 posts.

Also, you could maintain an array with the maximum post id of every author. Before fetching the top-100 posts of an author, you could check this array. This will avoid fetching/caching the posts of inactive users.

---

For Solution 1 , ordering by id will be a bit faster than ordering by date_time .