简体   繁体   中英

Puppeteer get all <a> href links

Hello I am trying to scrape a web page and return all of the links inside example of the html element:

<a href="#/item/2sDSXbG">
<a href="#/item/4ssaSXbG">
<a href="#/item/Sawd432">

Here is my code:

let links = [];
let elements2 = document.querySelectorAll('a');
  for (var element2 of elements2)
  links.push(element2.textContent);

After I return the value and print it I get an Error telling me that my variable is not defined My Error:

UnhandledPromiseRejectionWarning: ReferenceError: links is not defined

End Goal: My goal is to be able to be able to create an array of all the items in the list. I would than later parse the information so that it is just the text after /item/

It seems this is what you need to achieve your goal with puppeteer:

const hrefs = await page.evaluate(() => {
  let links = [];
  let elements2 = document.querySelectorAll('a');
  for (let element2 of elements2)
    links.push(element2.href);
  return links;
});

With $$eval:

let hrefs = await page.$$eval('a', as => as.map(a => a.href))

The anchors doesn't have any content. You need something like this

<a href="#/item/2sDSXbG">content1</a>
<a href="#/item/4ssaSXbG">content2</a>
<a href="#/item/Sawd432">content3</a>

The technical post webpages of this site follow the CC BY-SA 4.0 protocol. If you need to reprint, please indicate the site URL or the original address.Any question please contact:yoyou2525@163.com.

 
粤ICP备18138465号  © 2020-2024 STACKOOM.COM