简体   繁体   中英

Scroll down when scraping with phantom ( Node.js )

I'm using phantom to scrape a dynamic web page content and i need to scroll down to display all of the page content .

I've tried this await page.property('scrollPosition', { top: 1000, left: 0 }); but it didn't work for me and the content still the same.

this is my code :

(async function() {
    const instance = await phantom.create();
    const page = await instance.createPage();
    await page.on("onResourceRequested", function(requestData) {
      //console.info("Requesting", requestData.url);
    });

    const status = await page.open(
      "https://www.articles-epresse.fr/media/894eab75-c642-46a2-a1ba-b240c278ebbc?"
    );

    if (status == "success") {
      await page.property("scrollPosition", {
        top: 1000,
        left: 0
      });
    }
    const content = await page.property("content");

    console.log(content);
    var $ = cheerio.load(content);

Any idea ?

我只需要将top的值增加到3000这样就可以刮掉页面的其余部分

The technical post webpages of this site follow the CC BY-SA 4.0 protocol. If you need to reprint, please indicate the site URL or the original address.Any question please contact:yoyou2525@163.com.

 
粤ICP备18138465号  © 2020-2024 STACKOOM.COM