Puppeteer 和 Chromium 使用大型 CPU？

Question

我有一個 nodejs 腳本在我的 Macbook 24/7 上運行，每隔 20 分鍾左右它就會運行 Puppeteer 來解析網站的內容。 每隔兩天，風扇就會變得非常響亮，我看到 Chromium 正在使用大量 CPU 和 RAM。 有什么辦法可以解決這個問題嗎？

這是我正在運行的代碼：

const options = {
                args: [
                    '--no-sandbox',
                    '--disable-setuid-sandbox',
                    '--disable-dev-shm-usage',
                    '--disable-accelerated-2d-canvas',
                    '--no-first-run',
                    '--no-zygote',
                    '--single-process', 
                    '--disable-gpu'
                ],
                headless: true
            }

const browser = await puppeteer.launch(options);

const page = await browser.newPage();

await page.setUserAgent('Mozilla/5.0 (Macintosh; Intel Mac OS X 10_14_0) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/84.0.4147.125 Safari/537.36')

await page.setViewport({ width: 1920, height: 1080 });


await page.setRequestInterception(true);
page.on('request', (req) => {
if(req.resourceType() === 'stylesheet' || req.resourceType() === 'font' || req.resourceType() === 'image'){
    req.abort();
 } else {
      req.continue();
  }
});

await page.goto(url, { waitUntil: 'networkidle0' });


// ... do stuff

await page.close();
await browser.close();

Answer 1

我有一個想法，每 20 分鍾自動殺死並啟動 puppeteer。 我正在為這種方法使用瀏覽器池（集群）。

import cluster from 'cluster'
import * as os from 'os'
import * as process from 'process'
import * as puppeteer from 'puppeteer'

const launchOptions = {
    args: [
        '--no-sandbox',
        '--disable-setuid-sandbox',
        '--disable-dev-shm-usage',
        '--disable-accelerated-2d-canvas',
        '--no-first-run',
        '--no-zygote',
        '--single-process',
        '--disable-gpu'
    ],
    headless: true
}

let userAgentString = 'Mozilla/5.0 (Macintosh; Intel Mac OS X 10_14_0) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/84.0.4147.125 Safari/537.36'
let viewportSize = { width: 1920, height: 1080 }
let CPUsNum = os.cpus().length
CPUsNum = 1 // Set this number as how many browser page do you want to launch simultaneously

const runThis = async (processID) => {
    const browser = await puppeteer.launch(launchOptions)
    const page = await browser.newPage()
    await page.setUserAgent(userAgentString)
    await page.setViewport(viewportSize)
    await page.setRequestInterception(true)
    page.on('request', (request) => {
        let blockedResources = ['stylesheet', 'font', 'image']
        if (blockedResources.includes(request.resourceType())) {
            await request.abort()
        } else {
            await request.continue()
        }
    });
    await page.goto(url, { waitUntil: 'networkidle0' })
    // ... and do stuff
    // ... then close the page and browser and also kill the process
    await page.close()
    await browser.close()
    process.kill(processID)
    process.exit(0)
}

if (!cluster.isWorker) {
    console.log(`Primary ${process.pid} is running`)

    // Fork workers.
    for (let i = 0; i < CPUsNum; i++) {
        cluster.fork()
    }

    cluster.on('exit', (worker, code, signal) => {
        console.log(`worker ${worker.process.pid} died`) /*with code: ${code} and signal ${signal}*/
        cluster.fork() //lets recreate the worker that died
    })
} else {
    let processID = process.pid
    console.log(`Worker ${processID} started`)
    ;(async (processID: number) => {
        setInterval(await runThis(processID), 20 * 60 * 1000)
    })(processID)
}