Why does Puppeeteer cause my test suite to hang for 30 seconds when I use "waitForSelector" even though I'm calling "close" on the page and browser?

Question

I have a Node.js Mocha test suite (I've created a minimal reproduction based on the real world application I was trying to create an automated test for).

package.json :

{
  "name": "puppeteer-mocha-hang-repro",
  "version": "1.0.0",
  "description": "",
  "main": "index.js",
  "scripts": {
    "test": "echo \"Error: no test specified\" && exit 1"
  },
  "keywords": [],
  "author": "",
  "license": "ISC",
  "devDependencies": {
    "chai": "4.3.7",
    "express": "4.18.2",
    "mocha": "10.2.0",
    "puppeteer": "19.6.2"
  }
}

index.spec.js :

const expect = require('chai').expect;
const express = require('express');
const puppeteer = require('puppeteer');

const webServerPort = 3001;

describe('test suite', function () {
    this.timeout(10000);

    let webServer;
    let browser;

    beforeEach(async () => {
        // Start web server using Express
        const app = express();
        app.get('/', (_, res) => {
            res.send('<html>Hello, World from the <span id="source">Express web server</span>!</html>');
        });
        webServer = app.listen(webServerPort, () => {
            console.log(`Web server listening on port ${webServerPort}.`);
        });

        // Start browser using Puppeteer
        browser = await puppeteer.launch();
        console.log('Browser launched.');
    });

    afterEach(async () => {
        // Stop browser
        await browser.close();
        console.log('Browser closed.');

        // Stop web server
        await webServer.close();
        console.log('Web server closed.');
    });

    it('should work', async () => {
        const page = await browser.newPage();

        await page.goto(`http://localhost:${webServerPort}/`);
        console.log('Went to root page of web server via Puppeteer.');

        if (process.env['PARSE_PAGE'] === 'true') {
            const sel = await page.waitForSelector('#source');
            const text = await sel.evaluate(el => el.textContent);
            console.log('According to Puppeteer, the text content of the #source element on the page is:', text);
            expect(text).eql('Express web server');
        }

        await page.close();
        console.log('Page closed.');
    });
});

If I run the test suite with the command npx mocha index.spec.js , which causes lines 45-48 to be skipped, the test suite passes and the Mocha process ends quickly:

$ time npx mocha index.spec.js


  test suite
Web server listening on port 3001.
Browser launched.
Went to root page of web server via Puppeteer.
Page closed.
    ✔ should work (70ms)
Browser closed.
Web server closed.


  1 passing (231ms)


real    0m0.679s
user    0m0.476s
sys     0m0.159s

Note that it finished in 0.679s.

If I instead run it with the command PARSE_PAGE=true npx mocha index.spec.js , which causes none of my code to be skipped, the tests pass quickly but the process hangs for about 30 seconds:

$ time PARSE_PAGE=true npx mocha index.spec.js


  test suite
Web server listening on port 3001.
Browser launched.
Went to root page of web server via Puppeteer.
According to Puppeteer, the text content of the #source element on the page is: Express web server
Page closed.
    ✔ should work (79ms)
Browser closed.
Web server closed.


  1 passing (236ms)


real    0m30.631s
user    0m0.582s
sys     0m0.164s

Note that it finished in 30.631s.

I suspected that this meant I was leaving things open, forgetting to call functions like close . But, I am calling close on the Express web server, Puppeteer browser, and Puppeteer page. I tried calling close on the objects I use when I don't skip any of that code, which are sel and text . But if I try that, I get error messages telling me that those objects have no such functions.

System details:

$ node --version
v18.13.0
$ npm --version
9.4.0
$ lsb_release -a
No LSB modules are available.
Distributor ID: Ubuntu
Description:    Ubuntu 22.04.1 LTS
Release:        22.04
Codename:       jammy
$ uname -r
5.10.16.3-microsoft-standard-WSL

Answer 1

Update: this behavior is a regression fixed by #9612 and deployed as 19.6.3 . To fix the problem, upgrade to 19.6.3 (or downgrade to <= 19.6.0 if you're using an older Puppeteer for some reason).

See the original answer below.

I'm able to reproduce the hang, even without Mocha. It seems to be a bug in Puppeteer versions 19.6.1 and 19.6.2. Here's a minimal example:

const puppeteer = require("puppeteer"); // 19.6.1 or 19.6.2

const html = `<!DOCTYPE html><html><body><p>hi</p></body></html>`;

let browser;
(async () => {
  browser = await puppeteer.launch();
  const [page] = await browser.pages();
  await page.setContent(html);
  const el = await page.waitForSelector("p");
  console.log(await el.evaluate(el => el.textContent));
})()
  .catch(err => console.error(err))
  .finally(async () => {
    await browser?.close();
    console.log("browser closed");
  });

The culprit is page.waitForSelector , which seems to run its full 30 second default timeout even after resolving, somehow preventing the process from exiting. I've opened issue #9610 on Puppeteer's GitHub repo.

Possible workarounds:

Downgrade to 19.6.0.
Avoid using waitForSelector , since the data you want is in the static HTML (may not apply to your actual page though).
Call with page.waitForSelector("#source", {timeout: 0}) which seems to fix the problem, with the risk of stalling forever if used in a script (not a problem with mocha since the test will time out).
Call with page.waitForSelector("#source", {timeout: 1000}) which reduces the impact of the delay, with the risk of a false positive if the element takes longer than a second to load. This doesn't seem to stack, so if you use a 1-3 second delay across many tests, mocha should exit within a few seconds of all tests completing rather than the sum of all delays across all waitForSelector calls. This isn't practical in most scripts, though.
Run npx mocha --exit index.spec.js . Not recommended--this suppresses the issue.

I'm not sure if the behavior is specific to waitForTimeout or if it may apply to other waitFor -family methods.

As an aside, your server listen and close calls are technically race conditions, so:

await new Promise(resolve =>
  webServer = app.listen(webServerPort, () => {
    console.log(`Web server listening on port ${webServerPort}.`);
    resolve();
  })
);

and

await new Promise(resolve => webServer.close(() => resolve()));

System details:

$ node --version
v18.7.0
$ npm --version
9.3.0
$ lsb_release -a
No LSB modules are available.
Distributor ID: Ubuntu
Description:    Ubuntu 22.04.1 LTS
Release:    22.04
Codename:   jammy
$ uname -r
5.15.0-56-generic

I also confirmed the behavior on Windows 10.

Answer 2

I am not sure how much it will be helpful but you can try this:

if (process.env['PARSE_PAGE'] === 'true') {
   const sel = await page.waitForSelector('#source');
   const text = await page.evaluate(el => el.textContent, sel);
   console.log('According to Puppeteer, the text content of the #source element on the page is:', text);
   expect(text).eql('Express web server');
}

Also, check for global hooks!

Why does Puppeeteer cause my test suite to hang for 30 seconds when I use "waitForSelector" even though I'm calling "close" on the page and browser?

Question

2 answers

solution1
0 ACCPTED 2023-01-31 16:49:52

solution2
-1 2023-01-31 05:11:12

Why does Puppeeteer cause my test suite to hang for 30 seconds when I use "waitForSelector" even though I'm calling "close" on the page and browser?

Question

2 answers

solution1 0 ACCPTED 2023-01-31 16:49:52

solution2 -1 2023-01-31 05:11:12

solution1
0 ACCPTED 2023-01-31 16:49:52

solution2
-1 2023-01-31 05:11:12