understanding of microsoft testing software and usage

Question

This is a general question to understand the microsoft test software and its usage. Currently I am working with Microsoft Test Manager to create and run my manuall tests. In the project we have an Team Foundation Server which contains the project and we work on the code over visual studio. I don't know what team foundation server all contains. The unit tests are executed locally and seems not do be tracked by TFS. We work on a software that runs just on one computer and has only communication with another computer if this is required. So no web application or somethink like that

Now I would like to create automated UI tests and track them and the unit tests in TFS. Could please someone explain me what I have to do to accomplish this?

I have read that I need a lab environment to do automated tests. And for the environment I need additional software like test controller and test agent. But I could not install and get them to work.

Do I need to install the Test Controller on the TFS or where? Is Test Controller an own server software, included in TFS or like an Add on? Do every machine where I want to run and track test have the Test Agent software installed and configured?

I really hope someone can help me out.

Best Regards.

Answer 1

Test Automation in TFS and Visual Studio is a broad subject. There is a sort of natural progression for test Automation.

1. In Microsoft Test Manager you can do action recordings and use Deploying and testing web applications using Release Management to reduce the time spent on manual execution.

2. Then you can import those Action recordings into Visual Studio to generate code from them . Without any further installation of tools you can run these with the Visual Studio Test runner on your machine.

3. You can hand-code Unit tests and CodedUI tests using a Visual Studio Action Recorder . As with the tests generated from an Action Recording, you can play these back on your local machine without any further installation.

4. If you want Test Manager or Visual Studio to gather additional data from your application under test, you must install the Visual Studio Test Agent on the system under test. You can configure these test agents using a .TestSettings file . This allows you to run the tests from your machine, but gather additional information.

5. By associating your automated Tests to the Test Case work item and configuring Team Build for your application, MTM can execute the tests directly from MTM.

6. If you want the tests to run the tests as part of your build, you're going to need a Test Environment. A Standard Test Environment requires at least one Test Controller and the Test Agent on the machine that run the test as well as each machine you want to monitor during test execution. These environments can be configured and used from both automated builds as well as from Visual Studio or MTM.

7. If your development and test teams want to be able to quickly provision test labs using Hyper-V and virtualized labs, you can use the Test Agents in these environments as well to install and test your application.

Now, when you're ready to deploy your application to your non-development environments you can use Visual Studio Release Management to run your tests as part of the release management pipeline.

Since this is a very complex story, with all kinds of twists and turns I recommend you go through the Microsoft Patterns and Practices Guidance and the ALM Ranger guidance for testing. As well as some of the links referenced by Martin Hinshelwood.

• Testing for Continuous Delivery with Visual Studio 2012
• Test & Release Management Guide 2012
• Execute tests using Visual Studio Release Management
• Deploying and testing web applications using Release Management

So as you can see, the test agent and test controller are an essential element for certain scenarios, while its not needed for others.

The agent installers are available as a separate download from the Microsoft download site.

Answer 2

All of the Microsoft testing integration is designed to be built into the automated build within TFS. So typically you have build controllers, and build agents installed which run the tests every time the build happens.

You can also install test controllers and agents if you do not want it to run at build time, or if you need more distributed test runs. (Load testing, etc)

If you just want to run ui tests you can still do it locally based off of test data, but you won't be able to track test results as integrated.

Answer 3

There are a number of different tools available for testing in TFS.

You can create and run manual tests using MTM and if you are collecting action recordings you can generate coded ui tests that can be associated with your test case in MTM. This will push results back to the test case when it is run as part of a test automation run.

If your coders are creating unit tests then they can be run on a build server. However you should only be running tests that do not need an instance of your application.

http://nakedalm.com/create-release-management-pipeline-professional-developers/

If you have instance tested, integration or UI, then you should create a release pipeline in the release management tooling that Congress with TFS.

http://nakedalm.com/execute-tests-release-management-visual-studio-2013/

Once you have a deployment you can run increasing levels of instance testing untill your application pops out the pipeline.