What is the best way to filter out records from a large dataset

Question

I have around 500,000 users records in one table, and another table contains records of the users specifically blocked by other users (ie which user blocked which user). I am using the following query to filter the user records for the current user that don't contain the blocked users from blocked table.

SELECT * from demouser where id not in (select blockedid from demoblock where userid = 1 )

It is slow, taking around 2 seconds for the query to be executed. Is there any more efficient way to filter the records without comparing 500,000 user records to blocked table to remove blocked users from the results?

Answer 1

Create indexes on id and userid columns separately, so both the subquery and query will run faster.

Also, I recommend a slight modification:

select * from demouser du
where not exists(select 1 from demouser
                 where blockedid = du.id
                   and userid = 1)

Answer 2

I would write this as not exists :

select du.*
from demouser du
where not exists (select 1
                  from demoblock db
                  where db.blockedid = du.id and
                        db.userid = 1
                 );

For this query, you want an index on demoblock(blockedid, userid) .

Answer 3

Assuming that your demoblock table doesn't contain a lot of data, you can try with INNER JOIN . Example query below:

SELECT * FROM demouser INNER JOIN demoblock ON demouser.id=demoblock.id WHERE demoblock.id=1;