Rails fastest way to parse XML feed?

Question

I have about 50 XML feeds I need to parse and sort. I have done that using nokogiri, it parses the XML feeds on page load and creates a hash which I ilerate through. But is is really slow. Therefor I am looking for better solution.

Solutions I have thought:

1. Create a cron job that creates a static XML feed with all the 50 feeds parsed and sorted. Parse this XML feed with JS or nokogiri. Which is faster to parse it on User site or server side?

2. Break somehow the cron job XML feed up in parts for pageination.. The feed have for example 200-500 items and I only need to show the user about 8 items pr. page..

Answer 1

it parses the XML feeds on page load

Really bad idea. Unless you need super-fresh information and are willing to sacrifice some machine resources for it.

Fetch/parse them in a background process. Store results in a db (or file, whatever works) and serve your local content. This will be much faster.

Parse them in background even if they change very frequently. This way you don't burn CPU and load network by having several web workers do exactly the same work.