通过rails的mysql查询的最大大小是多少？

Question

I don't seem to be able to phrase this question well enough to google the answer.

What's the maximum length of a MySQL 5.1 query? Also, is there a relevant Rails 3 limit?

I have an array of 5,000+ email addresses, and need to grab the associated user ids from the db. My preference would be to do that all in one trip, like:

emails = get_emails # an array of strings
User.select(:id).where("email in (#{emails * ','})")

Even if this works for 5,000, I want to make sure that it will work someday for 50,000. So how big can queries be? And if they can't be that big, what's the next best way to get these ids? Just do the same thing in batches?

For this I care more about the officially supported limits than whatever happens to work on the current versions.

Answer 1

Depends on your max_allowed_packet setting in your MySQL configuration. That's the number of bytes that your MySQL server will accept.

I would agree with Pekka that the best way to see how large you can go is to try with fake values.

Also, you can write that query like this:

User.select(:id).where(:email => emails)

And Rails will automatically do the IN query.

Answer 2

You could do this in a 'sharded' manner -- and you would just be limited by the amount of memory you have...

For example store records 1..10,000 in Object1 and 10,001..20,000 in Object2 ... etc

In this sense there would be no limit on a rails query length