Puppeteer JS waiting to be able to select multiple drop downs

Question

Currently I'm trying to select multiple drop downs from a dynamic web page. Each selection has to be selected for me to be able to get a value I need.

When I load the page and try to select the drop downs like this:

  await page.evaluate(() => {
  document.querySelector('#paper > option:nth-child(2)').selected = true;
  document.querySelector('#color > option:nth-child(3)').selected = true;
  document.querySelector('#coating > option:nth-child(2)').selected = true;
  })

I am not able to select the drop downs. I think it is because the page has not loaded yet. To try to fix this I added anode module called sleep.

I import the sleep module like this:

const sleep = require('sleep');

and then I attempt to do sleep for one second in between each selector so it'll give it time to load.

  sleep.sleep(1);
  await page.evaluate(() => {
  document.querySelector('#paper > option:nth-child(2)').selected = true;
  sleep.sleep(1);
  document.querySelector('#color > option:nth-child(3)').selected = true;
  sleep.sleep(1);
  document.querySelector('#coating > option:nth-child(2)').selected = true;
  })

my problem is that the first sleep works. However the sleep.sleep(1) inside of the await page.evaluate says that sleep is undefined. I'm not sure why this is happening because const sleep is declared at the top of my document so it should be global scope. Can anyone guide me as to why the sleep doesn't work?

Answer 1

You should be using page.waitForSelector() to ensure that the elements have loaded before attempting to change the corresponding selected attributes:

await Promise.all([
  page.waitForSelector('#paper > option:nth-child(2)'),
  page.waitForSelector('#color > option:nth-child(3)'),
  page.waitForSelector('#coating > option:nth-child(2)'),
]);

await page.evaluate(() => {
  document.querySelector('#paper > option:nth-child(2)').selected = true;
  document.querySelector('#color > option:nth-child(3)').selected = true;
  document.querySelector('#coating > option:nth-child(2)').selected = true;
});

The reason that your sleep function does not work inside page.evaluate() is because the code inside the function is executed in the page DOM environment, not the Node.js/Puppeteer environment.

You can pass arguments to page.evaluate() , but the arguments must be serializable :

const example = 'Hello, world!';
const result = await page.evaluate(passed_argument => passed_argument, example);

console.log(result); // 'Hello, world!'