DeploymentItem and TestCleanup conflict in unit tests?

Question

I have an application that has many unit tests in many classes. Many of the tests have DeploymentItem attributes to provide required test data:

[TestMethod]
[DeploymentItem("UnitTesting\testdata1.xml","mytestdata")]
public void Test1(){
    /*test*/
}

[TestMethod]
[DeploymentItem("UnitTesting\testdata2.xml","mytestdata")]
public void Test1(){
    /*test*/
}

When tests are run individually, they pass. When all are run at once (For example, when I select "Run all tests in the current context"), some tests fail, because the DeploymentItem s left behind by other tests cause the tests to grab the wrong data. (Or, a test incorrectly use the files meant for another test that hasn't run yet.)

I discovered the [TestCleanup] and [ClassCleanup] attributes, which seem like the would help. I added this:

[TestCleanup]
public void CleanUp(){
    if(Directory.Exists("mytestdata"))
        Directory.Delete("mytestdata", true);
}

The trouble is, this runs after every test method, and it seems that it will delete DeploymentItems for tests that have not run yet. [ClassCleanup] would prevent this, but unfortunately, it would not run often enough to prevent the original issue.

From the MSDN documentation, it seems that DeploymentItem only guarantees that the files will be there before the test executes, but it is not more specific than that. I think I am seeing the following problem:

1. Deployment Item for test executes
2. (other stuff happens?)
3. Test cleanup from previous test executes
4. Next test executes
5. Test fails because files are gone

Does anyone know the execution order of the different test attributes? I've been searching but I haven't found much.

I have thought about having each deployment item use its own, unique folder for data, but this becomes difficult as there are hundreds of tests to go through.

Answer 1

The order of the test attributes are as follows:

1. Methods marked with the AssemblyInitializeAttribute.
2. Methods marked with the ClassInitializeAttribute.
3. Methods marked with the TestInitializeAttribute.
4. Methods marked with the TestMethodAttribute.

Part of the problem is that Visual Studio runs tests in a non-deterministic order(by default but this can be changed ) and multiple at a time. This means that you cannot delete the folder after each test.

---

In general, if you can avoid going to the disk for unit tests it will be much better. In general you don't want to have anything besides code that can break your tests.

Answer 2

I had a similar problem. In few tests i need to delete a deployed item - all tests pass when run individually, but failed when run in a playlist. My solution is ugly, but simple: use a different folder for every test .

For example:

    [TestMethod]
    [DeploymentItem("Resources\\DSC06247.JPG", "D1")]
    public void TestImageUploadWithRemoval()
    {
        // Arrange
        myDeployedImagePath = Path.Combine(TestContext.DeploymentDirectory, "D1", "DSC06247.JPG");
        // Act ...
    }

    [TestMethod]
    [DeploymentItem("Resources\\DSC06247.JPG", "D2")]
    public void TestImageUploadWithoutRemoval()
    {
        // Arrange
        myDeployedImagePath = Path.Combine(TestContext.DeploymentDirectory, "D2", "DSC06247.JPG");
        // Act...
    }