Visual test fail on gitlab pipeline because of font rendering

Question

I have an Angular project with Storybook as a design system. I want to integrate visual testing for it. I use Jest with puppeteer and image snapshot. Locally it is running fine but on GitLab, the pipeline is failing, because the font is rendered differently. I use a specific font but it is also included in the GitLab environment but it seems more stretched:

I also run the same docker container locally and on GitLab.

Here is my jest.config.js

process.env.JEST_PUPPETEER_CONFIG = 'node_modules/jest-puppeteer-docker/src/config.js';

module.exports = {
  preset: 'jest-puppeteer-docker',
  rootDir: '../',
  setupFilesAfterEnv: [
    '<rootDir>/.jest/jest-puppeteer.config.js'
  ],
  testMatch: [
    '<rootDir>/.storybook/visual-tests/*.visual-test.js'
  ]
};

And my jest-puppeteer.config.js

module.exports = {
  launch: {
    headless: true,
    args: [
      '--enable-font-antialiasing=false',
      '--font-render-hinting=medium',
      '--no-sandbox',
      '--disable-setuid-sandbox'
    ]
  }
};

It would be great if somebody could help me with this.

Answer 1

Had the same issue a while ago.

In my case, it was due to variation in OS of the pipeline agent (GitLab) and my local developer OS. Font can be rendered differently dependent of the OS.

Jest-snapshot issue

As for now I only believe that there are two ways around this issue:

1. Enable a minimal failure threshold on the jest-snapshot test. Not ideal, but dependent on the amount of text, it might work. In our case, we had a fair amount of text, which would require a failure threshold of approximately 1-2%. At this percentage, we started seeing other visual changes slip through the test threshold. We therefore investigated a second solution.

2. Run the jest-snapshot test with the same OS as the images are captured. On top of my head I see two solutions:

   2.a. Start up a VM of the same OS as the pipeline agents and use it for development.

   2.b. Include OS dependent images into the project.

I realize that none of these solutions are ideal, but I hope that they will help you find the most suitable way forward.

Cheers!

Answer 2

Even if you have a custom font, like let's say "Roboto", it will take some time to render in the container's "fresh browser without cached fonts".

A workaround is to use a timeout of, let's say 500ms to let the browser time enough to download and render the font. Another workaround would be to check that the font is loaded, thanks to fonts api