Getting Nokogiri to work with Delayed jobs

Question

I am trying to get Nokogiri to behave when using it with delayed jobs but haven't been very successful so far.

Basically I am trying to run a parsing task in the background, but when the background worker hits my perform method, it fails in the following line:

HTML_page = Nokogiri::HTML(open('http://www.mysite.com'))

The error message is:

Nokogiri::HTML::Document#inspect failed with ArgumentError: Requires a Node, NodeSet or String argument, and cannot accept a Delayed::Backend::ActiveRecord::Job.

This happens with both Delayed::Jobs.enqueue and delay methods.

If I try the line below in the console, I get the same error:

Nokogiri::HTML(open('http://www.mysite.com')).delay

It might be a silly oversight as I am fairly new to Ruby and Rails, so any help would be greatly appreciated.

Answer 1

Since Nokogiri "Requires a Node, NodeSet or String argument", why not give it one?

Instead of:

HTML_page = Nokogiri::HTML(open('http://www.mysite.com'))

try:

HTML_page = Nokogiri::HTML(open('http://www.mysite.com').read)

That will cause IO to read the file handle created by open and pass Nokogiri the string content of the URL being read.

An alternate way to help debug the problem, which I don't think lies within Nokogiri, is to split your command up a bit:

body = open('http://www.mysite.com').read
HTML_page = Nokogiri::HTML(body)