Combined sql query takes too much time

Question

My one query in MySQL takes too much time in executing. In this query, I use IN operator for fetching database from MySQL database.

My query :

SELECT *
FROM databse_posts.post_feeds
WHERE
    post_id IN (SELECT post_id FROM database_users.user_bookmarks where user_id=3) AND
    post_date < unix_timestamp();

In this, both individual queries takes very less time for execution like

SELECT post_id FROM database_users.user_bookmarks where user_id=3

takes around 400 ms max

and

SELECT * FROM databse_posts.post_feeds Where post_date < unix_timestamp();

takes 300 ms max

But combining both of queries in one using IN operator it takes around 6 to 7 secs. Why this taking too much time. I also write a different same type of queries but all those don't take that amount of time.

Answer 1

instead of a where IN (subselect) you could try a inner join on the subselect

SELECT *
FROM databse_posts.post_feeds
INNER JOIN (
    SELECT post_id 
    FROM database_users.user_bookmarks 
    where user_id=3
) T on T.post_id = post_feeds.post_id
AND
post_date < unix_timestamp();

and be sure you have proper index on post_feeds.post_id and user_bookmarks.user_id, user_bookmarks.post_id

Answer 2

My approach:

You need to create indexes for the fields post_feeds . post_id , user_bookmarks . post_id , user_bookmarks . user_id and post_feeds . post_date fields, then use INNER JOIN to let MySQL engine to manipulate the filtering and merging of rows in an efficient way:

SELECT
    pf.*
FROM
    databse_posts.post_feeds pf
    INNER JOIN database_users.user_bookmarks ub
        ON ( pf.post_id = ub.post_id )
WHERE
    ub.user_id = 3
    AND pf.post_date < unix_timestamp();

Answer 3

My rough guess here would be that the WHERE IN expression is doing something of which you might not be aware. Consider your full query:

SELECT *
FROM databse_posts.post_feeds
WHERE
    post_id IN (SELECT post_id FROM database_users.user_bookmarks where user_id=3) AND
    post_date < unix_timestamp();

MySQL has to check, for each record, each value of post_id and compare it to the list of post_id coming from the subquery. This is much more costly than just running that subquery once. There are various tricks available at MySQL's disposal to speed this up, but the subquery inside a WHERE IN clause is not the same as just running that subquery once.

If this hypothesis be correct, then the following query should also be in the range of 6-7 seconds:

SELECT *
FROM databse_posts.post_feeds
WHERE
    post_id IN (SELECT post_id FROM database_users.user_bookmarks where user_id=3)

If so, then we would know the source of slow performance.