在同步测试中使用 xUnit.net 将测试标记为异步

Question

Is there any benefit to setting the signature of tests such as async Task MyTest() with synchronous tests? Does xUnit.net run faster if you do? Or are they unnecessary?

Answer 1

Marking synchronous tests as async will make your tests run slower

xUnit asynchronous tests should only be used to test asynchronous methods. Arbitrarily marking synchronous test methods as asynchronous will actually make execution significantly slower .

Why would an async method be slower?

This happens because .NET does quite a bit more work to invoke an asynchronous task than it does to invoke a standard method. Marking any synchronous .NET method as async will make the execution of that method slower and xUnit test methods are not an exception to this rule.

To illustrate this point, I put together a simple dotnetfiddle with benchmarks comparing asynchronous and synchronous execution of a SHA1 hash computation. The significance of the execution penalty will vary depending on the duration of the core computation. In this particular example, running the synchronous version was consistently about 10 times faster than when the operation was invoked asynchronously . I'll include this code at the end of the answer, because I want to make a few other points first.

How xUnit runs your test

You can see how xUnit invokes test methods by examining the code in the TestInvoker and its subclass XunitTestInvoker . Basically, every test method is executed by an async method in the runner. xUnit uses reflection to invoke the test method. If the test method is async, then the xUnit runner method will wait for the test method task to finish.

• XunitTestInvoker.InvokeTestMethodAsync
• TestInvoker.InvokeTestMethodAsync

Speculation: Why you might be considering making your tests async and what to do instead

In general, performing asynchronous operations can improve performance because the work of the asynchronous operation can continue while the main thread does some other work. My guess is that your hope in asking this question is that xUnit would continue to do other work while the methods you marked async are running. While that's not what would happen, xUnit does provide a way to execute multiple tests concurrently. The appropriate technique to achieve this is to configure xUnit for running tests in parallel . Parallelism in xUnit test execution is orthogonal to the synchronization of your test methods, so this can be leveraged to more quickly execute both synchronous and asynchronous tests.

benchmarks of synchronous vs asynchronous execution of synchronous operations

using System;
using System.Diagnostics;
using System.Security.Cryptography;
using System.Threading.Tasks;

public class Program
{
    public static void Main() {

        TimeSpan
            averageSyncTime = TimeSpan.MinValue,
            averageAsyncTime = TimeSpan.MinValue;

        // init stopwatch
        var t = Stopwatch.StartNew();
        t.Stop();
        t.Reset();

        using(var algorithm = SHA1.Create()){
            var i = -1;
            var iMax= 10000;

            Action SyncOperationToTest = () => {
                var hash = algorithm.ComputeHash(BitConverter.GetBytes(i));
            };

            // start testing synchronous execution of synchronous operation
            t.Start();
            for(i = 1; i <= iMax; i++){
                SyncOperationToTest();
            }

            t.Stop();
            averageSyncTime = TimeSpan.FromTicks(t.ElapsedTicks/iMax);
            // stop testing synchronous execution of synchronous operation 

            t.Reset();

            // start testing asynchronous execution of synchronous operation
            t.Start();
            for(i = 1; i <= iMax; i++){
                Task.Run(SyncOperationToTest).Wait();
            }       
            t.Stop();
            averageAsyncTime = TimeSpan.FromTicks(t.ElapsedTicks/iMax);
            // stop testing asynchronous execution of synchronous operation 

            var howManyTimesSlowerIsAsync = Math.Round(averageAsyncTime.TotalMilliseconds/averageSyncTime.TotalMilliseconds, 1);

            Console.WriteLine("When called synchronously, the average operation time was {0}ms.", averageSyncTime.TotalMilliseconds);
            Console.WriteLine("When called in an artificial asynchronous wrapper, the average operation time was {0}ms.", averageAsyncTime.TotalMilliseconds);
            Console.WriteLine("Arbitrarily calling the example synchronous operation in an asynchronous wrapper was about {0} times slower than simply calling it synchronously.", howManyTimesSlowerIsAsync);
        }
    }
}

Answer 2

Unless you are testing asynchronous operations it is unnecessary to use the method signatures you mentioned. And Xunit will not run faster whether you have used it or not. The efficiency of your test operation depends on your logic.