How to scrape dynamically rendered data from a webpage with Selenium and Node.js?

Question

I am currently facing some difficulties scraping a website that is using react to render parts of its website and I'm not exactly sure why I can't scrape the data.

Here is the html of the website: image

What I would like to do is to grab the text of the button with data-test="shipItButton"

However when I run my code, I get this error:

(node:23294) UnhandledPromiseRejectionWarning: NoSuchElementError: no such element: Unable to locate element: {"method":"css selector","selector":"[data-test=shipItButton]"}

Here is my current code:

const webdriver = require("selenium-webdriver");
const By = webdriver.By;
const until = webdriver.until;

const driver = new webdriver.Builder()
  .forBrowser("chrome")
  .build();

let getData = async(url) => {
  await driver.get(url);
  let element = await driver.findElement(By.css('[data-test=shipItButton]'));
  await driver.wait(until.elementIsVisible(element), 15000)
    .then(element => getText())
    .then(text => console.log("What is text: ", text))
    .catch(err => console.log("Failed to get: ", err));
}
getData("https://www.target.com/p/animal-crossing-new-horizons-8211-nintendo-switch/-/A-76780148")

Any ideas on how to solve this?

Answer 1

修复这个区域。

By.css("button[data-test='shipItButton']")