在 RSelenium 中使用下拉值抓取表

Question

New to Selenium. I'm trying to scrape results from this table populated with dynamic data:

The Elements are as follows:

I wish I could add the link but it's password-protected and sharing would be a security issue. I'm having trouble finding something comparable to post as a suitable example and suggestions are welcomed.

For a previous table (also based on dynamic information, but not drop-downs) this code worked for me:

# WORKED FOR NON-DROPDOWN DYNAMIC TABLE
require(RSelenium)
pJS <- phantom()
Sys.sleep(5)
remDr <- remoteDriver(browserName = "phantomjs")
remDr$open()
url <- paste0("http://",un,":",pass,"@egauge", meter, ".egaug.es/57A4C/check.html")
remDr$navigate(url)
#remDr$screenshot(display = TRUE)
doc <- htmlParse(remDr$getPageSource()[[1]])
remDr$close()

readHTMLTable(doc)[[2]]

but if I use the same approach with the current table, it gives me back all the values in the drop-down box. My current attempt looks like this:

# NON-WORKING ATTEMPT FOR DROPDOWN DYNAMIC TABLE
require(RSelenium)

pJS <- phantom()
Sys.sleep(5)
remDr <- remoteDriver(browserName = "phantomjs")
remDr$open()
url <- paste0("http://",un,":",pass,"@egauge", meter, ".egaug.es/57A4C/settings.html")
remDr$navigate(url)
remDr$findElement("xpath", '//*[@id="regsDiv"]/div/table/tbody/tr[1]/td[3]/span[2]/select[1]')$selectTag()

But this only returns a list of Elements, options, and text. I need the selected option. Once this works I loop through with tr[] (from the xpath) and scrape all the selected values.

Thanks! Any help appreciated! I'm sorry I don't have a great working example.

EDIT: updated for response.

Answer 1

You can use the isElementSelected method:

remDr$navigate("http://nhb.gov.in/OnlineClient/MonthlyPriceAndArrivalReport.aspx")
elem <- remDr$findElement(
  using = "id", 
  value = "ctl00_ContentPlaceHolder1_ddlyear"
)
options <- elem$selectTag()
res <- vapply(options$elements, function(x) x$isElementSelected()[[1]], 
              logical(1))
res
> res
[1]  TRUE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE
[16] FALSE FALSE FALSE

UPDATE: The dev branch now adds a selected field to the selectTag method:

devtools::install_github("ropensci/RSelenium")

remDr$navigate("http://nhb.gov.in/OnlineClient/MonthlyPriceAndArrivalReport.aspx")
elem <- remDr$findElement(
  using = "id", 
  value = "ctl00_ContentPlaceHolder1_LsboxCenterList"
)
options <- elem$selectTag()
options$selected
> options$selected
[1] FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE
[16] FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE
[31] FALSE FALSE FALSE FALSE

# Now select "AGRA" and BANGALORE
myScript <- 
  "for (var i=0, iLen=arguments.length; i<iLen; i++) {
        arguments[i].selected = true;
        }"
remDr$executeScript(myScript, options$elements[c(2,5)])
newoptions <- elem$selectTag()
newoptions$selected
> newoptions$selected
[1] FALSE  TRUE FALSE FALSE  TRUE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE
[16] FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE
[31] FALSE FALSE FALSE FALSE