I'm trying to scrape the names of all the people who commented on a post in our Facebook group. I downloaded the file locally and am able to scrape the names of the people who commented plus the people who replied to those comments. I only want the original comments, not the replies... it seems like I have to exclude the UFIReplyList class but my code is still pulling all the names. Any help would be greatly appreciated. Thanks!
require 'nokogiri'
require 'pry'
class Scraper
@@all = []
def get_page
file = File.read('/Users/mark/Desktop/raffle.html')
doc = Nokogiri::HTML(file)
# binding.pry
doc.css(".UFICommentContent").each do |post|
# binding.pry
author = post.css(".UFICommentActorName").css(":not(.UFIReplyList)").text
@@all << author
end
puts @@all
end
end
Scraper.new.get_page
Traverse ancestors for every .UFICommentActorName
element, to reject those contained within a .UFIReplyList
element.
@authors_nodes = doc.css(".UFICommentActorName").reject do |node|
# extract all ancestor class names;
# beware of random whitespace and multiple classes per node
class_names = node.ancestors.map{ |a| a.attributes['class'].value rescue nil }
class_names = class_names.compact.map{ |names| names.split(' ') }
class_names = class_names.flatten.map(&:strip)
# reject if .UFIReplyList found
class_names.include?('UFIReplyList')
end
@authors_nodes.map(&:text)
The technical post webpages of this site follow the CC BY-SA 4.0 protocol. If you need to reprint, please indicate the site URL or the original address.Any question please contact:yoyou2525@163.com.