使用实验室管理来运行单元/集成测试

Question

We have a complete Lab Management environment running Coded UI tests in nightly builds. What we are trying to achieve is to run our integration tests (regular TestMethod() with SQL connections) just before all the Coded UI tests to verify that our db scripts are executed correctly and that there are no new changes causing any problems.

So far I have found a way to execute tests remotely through .testrunconfig. The problem we have with that approach is that it's not possible to choose a testcontroller connected to a team project so I guess that would be only useful for running tests on physical machines outside of Lab Management?

One option seem to be to create a Test Case for each integration test and that should run it together with the UI tests but it feels like it will be to much maintenance managing hundreds of test cases just to run the integration tests. Also it would be better to completely separate the test runs for the different kinds of tests.

Is there any easy way to achieve this that I have totally missed? Or do I have to modify the lab build template to deploy and run the tests?

Answer 1

I guess that would be only useful for running tests on physical machines outside of Lab Management?

If you run your tests remotely through .testrunconfig you have to connect the Test Agent to another Test Controller which is NOT connected to the team project. Unfortunately it is impossible for the environment which are running under the Lab Management, to my knowledge.

What about this approach:

1. Create an Ordered Test containing all you integration tests.
2. Create a new Test Case "Integration Tests" and automate it by the ordered test So you do not have to maintain hundreds of Test Cases. You could also create several Ordered Tests if you want to group the integration tests and then create a "main" Ordered Test containing them. This way it will be easier to analyze test results especially if you have a lot of tests.
3. Let the integration tests run as a part of your existing nightly build .
4. Create a new Build Definition which does not start a build but uses the last successful nightly build and let your CodedUI tests run using Lab Build Template .

This way you will have different test runs for the different kinds of tests.

The only drawback is that you have to "synchronize" these two builds... You could just schedule the second build later so you could be sure the fist build is done.

It's not really perfect, I know... but this way you could easily achieve your goal.

Answer 2

I am not sure if there is an alternative solution, but on the project I am currently working on we have both our Unit and Integration Test Assemblies set under the Process options (Process>Basic>AutomatedTest>TestAssembly) in our Nightly Build.
This was achieved through altering the Default Build Process Template (not the Lab Default) a bit, as you suggested (I thought this was standard, but it's been a while).