Is there an alternative for using a .testsettings file with TestCases and Microsoft Test Manager?

Question

We have a peculiar situation here that is causing our automated tests to fail on a newly created lab environment, using TFS 2012.

We've always had a bunch of 'unit' tests that tested our DAL code, which in turn uses the Enterprise Library Data Application Block to perform operations on the database. This was setup quite a few years ago, to enable our clients to choose either SqlServer or Oracle databases alongside our product, taking advantage of the DatabaseFactory class and all the supporting generic interfaces and classes in the entlib.data. I mentioned 'unit' like this because these are actually not pure unit tests but integration ones, seeing as they require a real database to work.

To test the same SQL code against both databases, we maintain two separate .config files inside a 'Resources' folder in our TFS project branch, pointing to our test databases:

• Resources\\SqlServer\\ConnectionStrings.config (SqlServer specific connection strings)
• Resources\\Oracle\\ConnectionStrings.config (Oracle specific connection strings)

In the root Resources folder, there are two accompanying .testsettings files, responsible for deploying files specific to each database:

• Resources\\SqlServer.testsettings (which deploys the SqlServer\\ConnectionStrings.config file)
• Resources\\Oracle.testsettings (which deploys the Oracle\\ConnectionStrings.config file)

Since the whole structure is in source control, the testsettings is able to find the .config files by using relative paths, allowing us to test everything without having to setup parameters manually. On devs machines, we always select the SqlServer.testsettings file when running the tests, so that they don't need to have the whole oracle environment installed to validate their changes before checking in the code. The Oracle side of the validation always occurred in our build process, where we actually test every method twice: first using the same SqlServer.testsettings used by the developers, and then using the Oracle.testsettings .

This way, we can setup our test assemblies' app.configs to redirect the connectionStrings node to an external file, like this:

<configuration>
  <connectionStrings configSource="ConnectionStrings.config"/>
  ...

When the tests are run, mstest copies the adequate ConnectionStrings.config file to the test's working folder, based on which .testsettings was used to initiate the run.

This was working fine until today, when I discovered that tests started through Microsoft Test Manager ignore the Visual Studio .testsettings files . Now I'm trying to run these same tests in our lab environment but the ConnectionStrings.config files are not deployed (understandably) and the tests fail.

How can we achieve this without using .testsettings files? After having huge headaches trying to setup oracle correctly in our new x64 build server, we disabled Oracle tests in the build definition. Now that we started setting up our lab environment, we thought about having one of the machines in it configured with our whole system using Oracle, enabling us to again run these 'unit tests' with oracle-specific connection strings to validate our queries. At the same time, we want to keep testing everything locally and on the build server using SqlServer also.

I think using [DeploymentItem] in this case is impossible, since it is meant for static files and not selectable, dynamic ones like our current setup.

Is there any equivalent to the .testsettings deployment process that we could use with TestCases inside MTM/Lab Env? On the Properties tab for our TestPlan, I can see the Automated Runs -> Test Settings option, but that only seems to allow deployment by specifying absolute paths (which will actually be resolved on the target machines). Is there a way to specify a relative path there, pointing to our ConnectionStrings.config files checked in on TFS? Maybe yet another alternative exists that I'm missing, perhaps using multiple build configurations?

Answer 1

Create separate build configurations for each of the server types by going into Configuration Manager and click New under Active solution configurations . Edit the project file and do something like this:

<PropertyGroup Condition="'$(Configuration)' == 'Oracle'">
  <appConfig>App.Oracle.Config</AppConfig>
</PropertyGroup>
<PropertyGroup Condition="'$(Configuration)' == 'SQL'">
  <appConfig>App.SQL.Config</AppConfig>
</PropertyGroup>

Then ensure you have the correct connection strings in each of the config files. You can then configure TFS to build using those build configurations.

More info on using PropertyGroup and Condition , MSBuild Configurations and MSBuild project properties