[英]How do I save the scraped data from Nokogiri to a Rails database?
I want to save the scraped data to the database so that I can implement search and sorting functionality on it. 我想将抓取的数据保存到数据库中,以便可以在其上实现搜索和排序功能。
I tried creating a new Rake task and updating attributes but for that I need to run rake fetch-data
every time the data is scraped. 我尝试创建一个新的Rake任务并更新属性,但是为此,我需要在每次rake fetch-data
都运行rake fetch-data
。
app/controller 应用/控制器
def show
url = @scrapper.url
data = Nokogiri::HTML(open(url))
@doc= data.css(".s-item-container")
end
app/views/show 应用/视图/显示
<% @doc.each do |item| %>
<tr>
<td><%= item.css(".s-access-title").text %></td>
<td><%= item.css(".s-price").text %></td>
<td><%= item.css("span+ .a-text-normal").text %></td>
</tr>
<% end %>
It didnt work. 它没有工作。 Is there any other way to achieve this? 还有其他方法可以做到这一点吗?
rake db:create
创建数据库rake db:create
rails g model Product title:string price:decimal rating:float
创建“产品”模型rails g model Product title:string price:decimal rating:float
doc = Nokogiri::HTML(open(@scrapper.url )) items = doc.css(".s-item-container") items.each do |item| Product.create!( title: item.css(".s-access-title").text.strip, price: item.css(".s-price").text.to_d, rating: item.css("span+ .a-text-normal").text.to_f) end
to prevent duplicates 防止重复
items.each do |item|
title = item.css(".s-access-title").text.strip
product = Product.find_or_initialize(title: title)
product.price = item.css(".s-price").text.to_d
product.rating = item.css("span+ .a-text-normal").text.to_f
product.save!
end
声明:本站的技术帖子网页,遵循CC BY-SA 4.0协议,如果您需要转载,请注明本站网址或者原文地址。任何问题请咨询:yoyou2525@163.com.