P/S: I change "networkidle0" to "networkidle2" and it works perfect, thanks to vsemozhebuty who help me in a comment of the topic.
I'm coding a web scraping using Express & puppeteer. Below code is a part of my code, it is trying to visit a page and reload if getting an error.
// getting itemid from many requests and do task below
let browser = await puppeteer.launch();
let page = await browser.newPage();
url = 'https://tiki.vn/' + itemid ;
await page.goto(url, {waitUntil: 'networkidle0'})
.catch(function (error) { throw new Error(error); page.reload({ waitUntil: ["domcontentloaded"]});
});
const tikiproduct = await page.evaluate( () => {
// Doing some actions to scrape data
});
await browser.close();
return new Promise( resolve => { resolve(tikiproduct); });
But i only visit the link successful 1-2 times. Other visits always throw error: " TimeoutError: Navigation timeout of 30000 ms exceeded ". I tried to set "timeout: 0" but the page is still pending without any errors. Sounds like last visit not finished or cleared but i don't know how to fix it. Hope you guys help me in this problem, thanks for reading.
Here the "TimeoutError" definiton on puppeteer docs https://pptr.dev/#?product=Puppeteer&version=v5.5.0&show=api-class-timeouterror
It is a very simple error but there is not enough code sample to say where is the problem.
The technical post webpages of this site follow the CC BY-SA 4.0 protocol. If you need to reprint, please indicate the site URL or the original address.Any question please contact:yoyou2525@163.com.