Puppeteer resize before screenshot is causing flaky visual diff tests

Question

I am using BackstopJS for visual regression testing. Backstop uses Puppeteer as the test engine. Some screenshots are inconsistent because the viewport is being resized when Puppeteer takes a screenshot. My app has animations whenever the viewport is resized. Is there any way around this?

I've tried increasing the window and viewport size so that no elements are offscreen and Puppeteer doesn't need to resize the window. I've also tried setting scrollheight on the window to 0 for both x and y.

Answer 1

Backstop.js has both readyEvent and delay options you can set to help with inconsistencies in CSS animations. They can be used separately or in combination to ensure that your animations have had time to finish before backstop.js takes its screenshots.

David Walsh has a great blog post about how to use readyEvent to trigger backstop.js. His example is is based on an Angular app, but the principle is the same.

He basically logs a message to the console after his animations have finished. Backstop.js waits until it sees the message in the console before it runs.

In my case, I had elements that slid in from the sides as the user scrolls to the bottom of the page. So I wrote a bit of javascript to detect when the user (or the backstop.js rendering engine) has scrolled to the bottom of the screen.

 $(window).scroll(function() { if($(window).scrollTop() + $(window).height() == $(document).height()) { console.log("bottom"); } }); 

Once the bottom of the screen was reached I used console.log("bottom") to tell backstop.js that it could proceed. So in my backstop.json file my readyEvent scenario looked like "readyEvent": "bottom" where the word "bottom" corresponded to my console.log message.

Note: You can make your console.log message say anything (David uses "backstop.ready") so long as the message matches the readyEvent value in your backstop.json scenarios section.

If you run into issues with your terminal timing out when running your tests, try setting the readyEvent value back to an empty string to see if readyEvent is causing the problem. If so, read this issue on github.