Puppeteer - 仅在存在时单击按钮

Question

I'm trying to create a webscraper in Node, using Puppeteer. My first challange (which I thought would be easy), it's pass by a pagination "Load more" button.

But, when I run the following code, Puppeteer click on all "Load more" and after click on a content, when I need to stop clicking.

Why this happens?

    let loadMore = true;

    while (loadMore) {
        selector = 'ul.pager > li > a.button';

        await page.waitForSelector(selector, { timeout: 600 }).then(() => {
            page.click(selector);
        }).catch(() => {
            loadMore = false;
        });
    }

Thx all!

Answer 1

I would do it like this. I think timeout doesn't make a difference here, if the await inside the if's condition is not enough then there could be other problem with the script.

let loadMore = true;
while (loadMore) {
  const selector = 'ul.pager > li > a.button';

  if ((await page.$(selector)) !== null) {
    await page.click(selector);
  } else {
    loadMore = false;
  }
}