Proper way to paginate large amount of data

Question

I am working on a project in Rails 3 that requires to paginate a very large amount of data. What will be the proper way to paginate large amount of data ( eg forum posts ) to maximize performace? Would it be too much workload for the server if I only fetch, lets say, 10 models from a collection and fetch again when going to the next page? Or will it be better just load everything and use javascript to paginate them?

Answer 1

Assuming you are looking for SQL solutions:

Pulling 10 objects will is not a issue for a server load. Issue is what query are you using for filtering them. If you use simple primary key for filtering them or some other efficient identifier, it should be completely ok. If you simply want to list pages one after another (and 10 items on each page), i do not see a problem. Just make sure, you fetch the columns you need (that is avoid unnecessary columns in you select).

You can use will_paginate or kaminari for pagination. Regardless of your collection size, they will fetch only the records you require (number of objects you want to pull on each page).

kaminari

A Scope & Engine based, clean, powerful, customizable and sophisticated paginator for Rails 3

will_paginate

Pagination library for Rails 3, Sinatra, Merb, DataMapper, and more