NUnit3.testadapter is included in code coverage

Question

I faced a problem in Azure DevOps today. I'm trying to set up a test execution pipeline with code coverage enabled and everything works fine except one particular thing related to NUnit. When I run Code Coverage on my local Visual Studio instance, everything is perfect and results include only 2 targeted dll files (product + test). But when running in a build pipeline, I get nunit.framework.dll and nunit3.testadapter.dll in coverage too, which leads to wrong final coverage percentage.

Has anybody faced same issue?

Appreciate any valid points.

Answer 1

What you observe is correct, and is a bug in the VSTest task in Azure Devops. When VSTest runs within Visual Studio, it runs by what is defined in the solution file, but when it runs in the Azure Devops (cloud or server) it runs by what assemblies exists in the location. If those assemblies have pdb files they will be included by default.
The current workaround is to add a runsettings file, and add the two nunit dlls to a ModuleExclude section there.

<ModulePaths>
  <Exclude>
   <ModulePath>NUnit.Framework.dll</ModulePath>
   <ModulePath>NUnit3.TestAdapter.dll</ModulePath>
  </Exclude>
</ModulePaths>

Information on how to do this is in the following two documents:
Configure unit tests by using a .runsettings file

Customize code coverage analysis

You can also add and use the item template for runsettings to get the correct blocks.

The VSTest task has a separate field for declaring the runsettings file to use.