如何使用Nokogiri抓取网站并将数据输出到xml文件？

Question

I've been trying to scrape data using Nokogiri and HTTParty and can scrape data off a website successfully and print it to the console but I can't work out how to output the data to an xml file in the repo.

Right now the code looks like this:

class Scraper

  attr_accessor :parse_page

  def initialize
    doc = HTTParty.get("https://store.nike.com/gb/en_gb/pw/mens-nikeid-lifestyle-shoes/1k9Z7puZoneZoi3?ref=https%253A%252F%252Fwww.google.com%252F")
    @parse_page ||= Nokogiri::HTML(doc)
  end

  def get_names
    item_container.css(".product-display-name").css("p").children.map { |name| name.text }.compact
  end

  def get_prices
    item_container.css(".product-price").css("span.local").children.map { |price| price.text }.compact
  end

  private

  def item_container
    parse_page.css(".grid-item-info")
  end

  scraper = Scraper.new
  names = scraper.get_names
  prices = scraper.get_prices

  (0...prices.size).each do |index|
    puts " - - - Index #{index + 1} - - -"
    puts "Name: #{names[index]} | Price: #{prices[index]}"
  end

end

I've tried changing the .each method to include a File.write() but all it ever does is write the last line of the output into the xml file. I would appreciate any insight as to how to parse the data correctly, I am new to scraping.

Answer 1

I've tried changing the .each method to include a File.write() but all it ever does is write the last line of the output into the xml file.

Is the File.write method inside the each loop? I guess what's happening here is You are overwriting the file on every iteration and that's why you are seeing only the last line.

Try putting the each loop inside the block of the File.open method like:

File.open(yourfile, 'w') do |file|
  (0...prices.size).each do |index|
    file.write("your text")
  end
end

I also recommend reading about the Nokogiri::XML::Builder and then saving it's output to the file.