Mysql: Why is WHERE IN much faster than JOIN in this case?

Question

I have a query with a long list (> 2000 ids) in a WHERE IN clause in mysql (InnoDB):

SELECT id 
FROM table 
WHERE user_id IN ('list of >2000 ids')

I tried to optimize this by using an INNER JOIN instead of the wherein like this (both ids and the user_id use an index):

SELECT table.id 
FROM table 
INNER JOIN users ON table.user_id = users.id WHERE users.type = 1

Surprisingly, however, the first query is much faster (by the factor 5 to 6). Why is this the case? Could it be that the second query outperforms the first one, when the number of ids in the where in clause becomes much larger?

Answer 1

This is not Ans to your Question but you may use as alternative to your first query, You can better increase performance by replacing IN Clause with EXISTS since EXISTS performance better than IN ref : Here

SELECT id 
FROM table t
WHERE EXISTS (SELECT 1 FROM USERS WHERE t.user_id = users.id)

Answer 2

This is an unfair comparison between the 2 queries.

• In the 1st query you provide a list of constants as a search criteria, therefore MySQL has to open and search only table and / or 1 index file.

• In the 2nd query you instruct MySQL to obtain the list dynamically from another table and join that list back to the main table. It is also not clear, if indexes were used to create a join or a full table scan was needed.

To have a fair comparison, time the query that you used to obtain the list in the 1st query along with the query itself. Or try

SELECT table.id FROM table WHERE user_id IN (SELECT users.id FROM users WHERE users.type = 1)

The above fetches the list of ids dynamically in a subquery.