通过RSelenium单击按钮

Question

I am attempting to scrape REI's Reviews (Hammocks) using Rselarium and Rvest. I want to hit the button at the bottom x amount of times so I can scrape all the reviews. I'm a little lost. Here's what I have so far. If you know too, how to preview in the finder what you're doing (not screen print) that would be cool. Thanks Stack Community.

    replicate(100,
          {
remDr$navigate("https://www.amazon.com/Eagles-Nest-Outfitters-DoubleNest-Portable/product-reviews/B00K30GXK8/ref=cm_cr_dp_d_show_all_btm?ie=UTF8&reviewerType=all_reviewshttps://www.amazon.com/Eagles-Nest-Outfitters-DoubleNest-Portable/product-reviews/B00K30GXK8/ref=cm_cr_dp_d_show_all_btm?ie=UTF8&reviewerType=all_reviews")
webElem <- remDr$findElement("css", "body")
webElem$sendKeysToElement(list(key = "end"))
morereviews <- remDr$findElement(using = 'css selector', ".a-last a")
morereviews$clickElement
Sys.sleep(4)

reviews <- xml2::read_html(remDr$getPageSource()[[1]])%>%
  rvest::html_nodes(".review-text")%>%
  dplyr::data_frame(reviews = .)
})

Answer 1

Try this:

# Click the Load More button
replicate(100,
          {
            # scroll down
            webElem <- remDr$findElement("css", "body")
            webElem$sendKeysToElement(list(key = "end"))
            # find button
            morereviews <- remDr$findElement(using = 'css selector', "#BVRRContainer div.bv-content-pagination-container button")
            # click button
            morereviews$clickElement()
            # wait
            Sys.sleep(4)
          })

# Scrap the reviews
reviews <- xml2::read_html(remDr$getPageSource()[[1]])%>%
  rvest::html_nodes("#BVRRContainer div.bv-content-summary-body-text") %>%
  rvest::html_text() %>%
  dplyr::data_frame(reviews = .)
reviews