ActiveRecord speed up `where.not` query

Question

I have a query that is taking getting significantly slowed down when an not clause is added to the query.

# query is 13.6ms
FbGroupApplication.select(
:fb_id
).group(:fb_id
).having('count(fb_id) <= 6'
)


# 3,468ms with `where.not` clause. the array is large (5-10k)
FbGroupApplication.where.not(
fb_id: ids
).select(:fb_id
).group(:fb_id
).having('count(fb_id) <= 6'
)

I added an index on fb_id . However, one thing to note is that it's a text field and the values are about 12 character strings of numbers

How can I speed up this query?

Answer 1

Assuming that the ids are variable and due to the big number of them I guess your best approach would be to create a temporary table and insert the ids you want to exclude there.

Then query using LEFT JOIN, SELECT the ids that don't exist on the temporary table and DROP the table afterwards