在测试资源管理器中运行“全部运行”时是否可以排除测试项目

Question

I have some Specflow(3.9.40)-based UI test using Selenium Webdriver and NUnit(3.13.2) Framework.

I would like these tests to not run when I hit 'Run All' in the Test Explorer.

In my StepDefinition class I have tried adding the Explicit Attribute:

[TestFixture, Explicit("Only run when called explicitly")]
[Binding]
public class LoginStepDefinitions
{ ...

, but for some reason it does not seem to work - The UI Tests still run.

I imagine there may be something I can add to the [BeforeScenario] in my HookInitialization class, but I am stumped:

[BeforeScenario]
    public void FirstBeforeScenario()
    {
        var seleniumDriver = new SeleniumDriver(_scenarioContext);
        _scenarioContext.Set(seleniumDriver, "SeleniumDriver");
    }

Is there a way for me to tell Visual Studio Test Explorer to ignore the UI Tests or its folder?

Answer 1

The basic problem is that NUnit's Explicit setting is unique among the frameworks supported by Test Explorer, which knows nothing of this concept of "explicitness."

If Test Explorer calls the NUnit adapter telling it to run all, then NUnit will honor the Explicit setting. However, in some situations, it simply passes a list of all the tests to NUnit. In that case, all tests appear (to NUnit) to have been run explicitly.

Which way Test Explorer calls NUnit is dependent on the version you are running and it's only possible to tell how it calls by debugging or by observing the outcome. To add to the confusion, some versions of the NUnit3 adapter use heuristics to try to guess whether an explicit test was called explicitly. This doesn't solve your problem but should make it easier to understand what is going on.

The best workaround, if you are using TestExplorer, is to avoid Explicit and to set up categories, which control the running of the tests. Fortunately, TestExplorer and NUnit have pretty much the same understanding of categories.

Answer 2

You can filter tests first in Test Explorer , then the "Run All" button is scoped to only those tests in the Test Explorer panel. If you desire to run all tests except those in a certain project, enter this into the test filter search box in the Test Explorer panel:

-project:NameOfTheTestProject

If your test project has space characters in the name:

-project:"Name of the test project"

Note the - character before the word project . This is crucial. This character tells Visual Studio Test Explorer to invert your filter criteria — it is used to exclude tests in a certain project.

If you have these UI tests intermingled with other kinds of tests, obviously filtering out an entire project is not desirable. Consider placing the UI tests in a specific folder and filter on the FullName instead. For example, if your SpecFlow tests exist in a folder called "Specs", then omit those tests using:

-FullName:Specs

Again, include the - character (with no spaces) before the FUllName criteria to exclude tests in that namespace. This works with SpecFlow because the namespaces for C# classes generated from .feature files follow the folder structure of your test project, just like adding a .cs file does.

This technique should work with any unit test provider, not just NUnit.