Unit testing async method: How to explicitly assert that the internal task was cancelled

Question

I was recently writing an async method that calls a external long running async method so I decided to pass CancellationToken enabling cancellation. The method can be called concurrently.

Implementation has combined exponential backoff and timeout techniques described in Stephen Cleary 's book Concurrency in C# Cookbook as follows;

/// <summary>
/// Sets bar
/// </summary>
/// <param name="cancellationToken">The cancellation token that cancels the operation</param>
/// <returns>A <see cref="Task"/> representing the task of setting bar value</returns>
/// <exception cref="OperationCanceledException">Is thrown when the task is cancelled via <paramref name="cancellationToken"/></exception>
/// <exception cref="TimeoutException">Is thrown when unable to get bar value due to time out</exception>
public async Task FooAsync(CancellationToken cancellationToken)
{
    TimeSpan delay = TimeSpan.FromMilliseconds(250);
    for (int i = 0; i < RetryLimit; i++)
    {
        if (i != 0)
        {
            await Task.Delay(delay, cancellationToken);
            delay += delay; // Exponential backoff
        }

        await semaphoreSlim.WaitAsync(cancellationToken); // Critical section is introduced for long running operation to prevent race condition

        using (CancellationTokenSource cancellationTokenSource = CancellationTokenSource.CreateLinkedTokenSource(cancellationToken))
        {
            cancellationTokenSource.CancelAfter(TimeSpan.FromMilliseconds(Timeout));
            CancellationToken linkedCancellationToken = cancellationTokenSource.Token;

            try
            {
                cancellationToken.ThrowIfCancellationRequested();
                bar = await barService.GetBarAsync(barId, linkedCancellationToken).ConfigureAwait(false);

                break;
            }
            catch (OperationCanceledException) when (!cancellationToken.IsCancellationRequested)
            {
                if (i == RetryLimit - 1)
                {
                    throw new TimeoutException("Unable to get bar, operation timed out!");
                }

                // Otherwise, exception is ignored. Will give it another try
            }
            finally
            {
                semaphoreSlim.Release();
            }
        }
    }
}

I wonder if I should write a unit test that explicitly asserts that the internal task barService.GetBarAsync() is cancelled whenever FooAsync() is cancelled. If so how to implement it cleanly?

On top of that, should I ignore implementation details and just test what client/caller is concerned as described in method summary (bar is updated, cancel triggers OperationCanceledException , timeout triggers TimeoutException ).

If not, should I get my feet wet and start implementing unit tests for following cases:

1. Testing it is thread-safe (monitor acquired only by single thread at a time)
2. Testing the retry mechanism
3. Testing the server is not flooded
4. Testing maybe even a regular exception is propagated to caller

Answer 1

I wonder if I should write a unit test that explicitly asserts that the internal task barService.GetBarAsync() is cancelled whenever FooAsync() is cancelled.

It would be easier to write a test that asserts that the cancellation token passed to GetBarAsync is cancelled whenever the cancellation token passed to FooAsync is cancelled.

For asynchronous unit testing, my signal of choice is TaskCompletionSource<object> for asynchronous signals and ManualResetEvent for synchronous signals. Since GetBarAsync is asynchronous, I'd use the asynchronous one, eg,

var cts = new CancellationTokenSource(); // passed into FooAsync
var getBarAsyncReady = new TaskCompletionSource<object>();
var getBarAsyncContinue = new TaskCompletionSource<object>();
bool triggered = false;
[inject] GetBarAsync = async (barId, cancellationToken) =>
{
  getBarAsyncReady.SetResult(null);
  await getBarAsyncContinue.Task;
  triggered = cancellationToken.IsCancellationRequested;
  cancellationToken.ThrowIfCancellationRequested();
};

var task = FooAsync(cts.Token);
await getBarAsyncReady.Task;
cts.Cancel();
getBarAsyncContinue.SetResult(null);

Assert(triggered);
Assert(task throws OperationCanceledException);

You can use signals like this to create a kind of "lock-step".

---

Side note: in my own code, I never write retry logic. I use Polly , which is fully async -compatible and thoroughly tested. That would reduce the semantics that need to be tested down to:

1. The CT is passed through (indirectly) to the service method, resulting in OperationCanceledException when triggered.
2. There is also a timeout, resulting in TimeoutException .
3. Execution is mutex'ed.

(1) would be done just like the above. (2) and (3) are less easy to test (for proper tests, requiring either MS Fakes or abstractions for time/mutex). There is definitely a point of diminishing returns when it comes to unit testing, and it's up to you how far you want to go.

Answer 2

Thanks to Stephen Cleary for the nod to Polly retry. Perhaps of interest to future readers, all the functionality in the original poster's code sample could now be built from ready-made Polly primitives which are already unit-tested:

• Timeout policy for timeout by timing-out cancellation token (including combining with a user-supplied cancellation token)
• Bulkhead policy for parallelization throttling/mutual exclusion
• WaitAndRetry for retry, including cancellation during waits
• PolicyWrap to combine.

All Polly policies are fully unit-tested , sync and async-compatible, thread-safe for concurrent executions, and have pass-through cancellation support.

So, the intention of the original code could be achieved something like:

Policy retry = Policy.Handle<WhateverExceptions>().WaitAndRetryAsync(RetryLimit, retryAttempt => TimeSpan.FromMilliseconds(250 * Math.Pow(2, retryAttempt)));
Policy mutex = Policy.BulkheadAsync(1);
Policy timeout = Policy.TimeoutAsync(/* define overall timeout */);

bar = await timeout.WrapAsync(retry).WrapAsync(mutex).ExecuteAsync(ct => barService.GetBarAsync(barId, ct), cancellationToken);

---

I'll add some comments about unit-testing (the OP's original question) to the comments to Stephen's (much more relevant) answer to that.