简体   繁体   中英

How can I capture all links in a page with Puppeteer?

trying capturing all the <a> in a page

the console.log returns undefined, but i can't understand why is this const anchors = Array.from(document.querySelectorAll(sel)); correct?

const puppeteer = require('puppeteer');

(async () => {
  const browser = await puppeteer.launch({
                                            headless: false,
                                            userDataDir: "C:\\Users\\johndoe\\AppData\\Local\\Google\\Chrome\\User Data\\Default"
                                        });
  const page = await browser.newPage();
  await page.setViewport({
    width: 1920,
    height: 1080,
    deviceScaleFactor: 1,
  });
  await page.goto('https://www.facebook.com/groups/632312010245152/members');
  
  //https://github.com/puppeteer/puppeteer/blob/main/examples/search.js
  let membri = await page.evaluate((sel) => { 
    const anchors = Array.from(document.querySelectorAll(sel));
    return anchors;
  }, 'a');
  console.log(membri);
})();
const findLinks = await page.evaluate(() =>
  Array.from(document.querySelectorAll("a")).map((info) => ({
    information: info.href.split()
  }))
);
links = [];
findLinks.forEach((link) => {
  if (link.information.length) {
    links.push(link.information);
  }
});
await console.log(links);
await page.close();
return links;

Not sure if this is the most optimized solution, but it works. If you could message me a cleaned version of this code I would highly appreciate that:)

const arrayList = await page.evaluate(() => { const nodeListLinks = document.querySelectorAll('a'), array = [...nodeListLinks], list = array.map(({href}) => ({href})) return arrayList }) console.log(arrayList)

The technical post webpages of this site follow the CC BY-SA 4.0 protocol. If you need to reprint, please indicate the site URL or the original address.Any question please contact:yoyou2525@163.com.

 
粤ICP备18138465号  © 2020-2024 STACKOOM.COM