How can I select a div in Javascript with only a class using puppeteer

Question

I am trying to setup a bot that will click a button on a webpage for a project. Now this works on other webpages fine, however, on the page I want to use it on, the developers apparently thought it would be a brilliant idea to have almost no ids at all. So I am stuck with trying to figure out how to select a div by its class (the only available selector it has). Yes I have tried using Xpath: both times return null or an error. Here is the working code if it is on another webpage:

const puppeteer = require('puppeteer');

async function scrapeProduct(url) {
  const browser = await puppeteer.launch();
  const page = await browser.newPage();
  await page.goto(url);
  await page.click('#someid');

  await page.screenshot({ 
    path: "success.png", 
    fullPage: true
  });
  await browser.close();
}

scrapeProduct('someurl');

This is me testing Xpath which works but again this doesnt work for what I am wanting to do (unsure if this is the right way anyways - seems somewhat roundabout and I dont know how you would click with this):

async function scrapeProduct(url) {
  const browser = await puppeteer.launch();
  const page = await browser.newPage();
  await page.goto(url);
  const [el] = await page.$x('//*[@id="btt-btn"]');
  const classID = await el.getProperty('textContent');
  const classIDText = await classID.jsonValue();
  console.log(classIDText);

  await page.screenshot({ 
    path: "success.png", 
    fullPage: true
  });
  await browser.close();
}

scrapeProduct('someurl');

The error I get talks about no node being found with that selector, which in this case is '.DropdownSelectInput__SelectBoxText-sc-1ssquc7-0 bnoarO'

Does anyone have any thought? I have tried multiple different selectors and methods and cannot work out how I might do this.

Answer 1

I had the similar issue before. Best way to select your div is following:

In the same HTML tree, select an element with a certain attribute. It can be an attribute name like data-at or id or anything that doesnt change at each load. Once you manage to select that element, you can go up and down in the tree with element.parentNode or node.lastChild or node.firstChild etc. This would be the only way to do it. Possibly website that you are trying to manipulate uses something that generate classes. So, no human being gave that classnames.