rvest Web Scraping: Getting http form from site with web-shell-spinner

Question

I'm attempting to extract a form from the following site: 'https://app.qbo.intuit.com/app/login?loadCustomerAssistanceAssets=us&product=qbOnline'

However, it seems that these is a web-shell-spinner preventing me from doing so.

# Get url
main_url <- "https://app.qbo.intuit.com/app/login?loadCustomerAssistanceAssets=us&product=qbOnline"

# Connection
session.1 <- session(main_url)
http_response <- html_form(session.1)

When running just the html_form(session.1) line I get an empty list:

Im pretty sure it has something to do with the web-spinner on the QuickBooks site:

Any thoughts on how I can extract the form and POST HTTP request to login?

Thanks,

Answer 1

Here is an example of how you can navigate to the page, enter your username in the form and click on the "sign-in button".

library(RSelenium)

url <- "https://app.qbo.intuit.com/app/login?loadCustomerAssistanceAssets=us&product=qbOnline"
shell('docker run -d -p 4445:4444 selenium/standalone-firefox')
remDr <- remoteDriver(remoteServerAddr = "localhost", port = 4445L, browserName = "firefox")
remDr$open()
remDr$navigate(url)

web_Obj_Username <- remDr$findElement("xpath", '//*[@id="ius-signin-userId-input"]')
web_Obj_Username$sendKeysToElement(list("myusername"))

web_Obj_Signin <- remDr$findElement("xpath", '//*[@id="ius-identifier-first-submit-btn"]')
web_Obj_Signin$clickElement()