Nokogiri doesn't work using css selector

Question

My code:

require 'rubygems'
require 'nokogiri'
require 'open-uri'

PAGE_URL = "http://www.whoscored.com/Teams/1799/Fixtures/Spain-Almeria"

page = Nokogiri::HTML(open(PAGE_URL))

CSS_SELECTOR = "a.match-link.match-report.rc"

links = page.css(CSS_SELECTOR)

puts links.length   # => RESULT = 0
puts links[0].text   # => RESULT = none
puts links[0]["href"] # => RESULT = ./read_stats.rb:15:in `<main>': undefined method     `text' for nil:NilClass (NoMethodError)

The results should be:

1
Match Report
/Matches/738463/MatchReport

But my results are:

0
./read_stats.rb:15:in `<main>': undefined method `text' for nil:NilClass (NoMethodError)

It doesn't work, and I dont see the problem...

Thanks.

Answer 1

You need to render the page before you can scrape it.

require 'watir-webdriver'
require 'nokogiri'

$browser = Watir::Browser.start "http://www.whoscored.com/Teams/1799/Fixtures/Spain-Almeria"

$page_html = Nokogiri::HTML.parse($browser.html)

$page_html.css("td[@class='toolbar right']").each do |me|
  print "#{me.count}\n#{me.text}\n#{me.css("a").map{|link| link['href']}[0]}\n\n"
end

Try out watir.com