Nokogiri parsing table with no html element

Question

I have this code that attempts to go to a URL and parse 'li' elements into an array. However I have run into a problem when trying to parse anything that is not in a 'b' tag.

Code:

url = '(some URL)'
page = Nokogiri::HTML(open(url))
csv = CSV.open("/tmp/output.csv", 'w')

page.search('//li[not(@id) and not(@class)]').each do |row|
  arr = []
  row.search('b').each do |cell|
    arr << cell.text
  end
  csv << arr
  pp arr
end

HTML:

<li><b>The Company Name</b><br>
The Street<br>
The City, 
The State 
The Zipcode<br><br>
</li>

I would like to parse all of the elements so that the output would be something like this:

["The Company Name", "The Street", "The City", "The State", "The Zip Code"],
["The Company Name", "The Street", "The City", "The State", "The Zip Code"],
["The Company Name", "The Street", "The City", "The State", "The Zip Code"]

Answer 1

require 'nokogiri'

def main
  output = []
  page = File.open("parse.html") {|f| Nokogiri::HTML(f)}
  page.search("//li[not(@id) and not (@class)]").each do |row|
    arr = []
    result = row.text
    result.each_line { |l|
      if l.strip.length > 0
        arr << l.strip
      end
    }
    output << arr
  end
  print output
end

if __FILE__ == $PROGRAM_NAME
  main()
end

Answer 2

I ended up finding the solution to my own question so if anyone is interested I simply changed

row.search('b').each do |cell|

into:

row.search('text()'.each do |cell|

I also changed

arr << cell.text

into:

arr << cell.text.gsub("\n", '').gsub("\r", '') 

in order to remove all the \\n and the \\r that were present in the output.

Answer 3

Based on your HTML I'd do it like:

require 'nokogiri'

doc = Nokogiri::HTML(<<EOT)
<ol>
<li><b>The Company Name</b><br>
The Street<br>
The City, 
The State 
The Zipcode<br><br>
</li>
<li><b>The Company Name</b><br>
The Street<br>
The City, 
The State 
The Zipcode<br><br>
</li>
</ol>
EOT

doc.search('li').map{ |li|
  text = li.text.split("\n").map(&:strip)
}
# => [["The Company Name",
#      "The Street",
#      "The City,",
#      "The State",
#      "The Zipcode"],
#     ["The Company Name",
#      "The Street",
#      "The City,",
#      "The State",
#      "The Zipcode"]]