有没有Queue的替代方法，可让您一次.pop多个结果？

Question

I'm using Ruby's Queue on my Sinatra server to send tasks to a worker thread to then deal with them and save them in a database of my choosing. One of the databases happens to be Redis, which supports some really cool pipelinening .

My question is: is there a way to modify the Queue class so that .pop can pop more than one item at a time like normal Array#pop , à la [1,2,3,4,4,5].pop(3) , so that I can pipeline the results?

Also, what happens to a Heroku instance if a Queue becomes too large? A warning, and a restart - some lost data, correct?

Answer 1

How about:

def pop_queue(q, n)
  ary = []
  n.times { ary << q.pop }
  ary
end

You don't want to monkey-patch a core library. If other code calls it and isn't aware of the change in arity or parameter type, your overall application will blow up.

...if your data becomes too large? You are the programmer. Aren't you supposed to program defensively to avoid those situations?

If you have a possible situation where you could use all available space, then write your code to not suck up RAM and instead use a database. At a minimum use a disk-based SQLite database and something like Sequel to talk to it.

The beauty of a database is multiple threads can talk to it, just don't read and write to the same row. Some DBMs support row locking and others support table locking. You'll have to write your code to take advantage of that and treat the table like your queue.

If you want the best performance, use a message queue, like AMQP or RabbitMQ instead of the database, but that's starting to talk about more complex code and bigger hosting iron.