How do I unit test that a file uploads to Amazon S3 in my C# project?

Question

I've started writing unit tests for an Amazon S3 container for files. I've never written tests for S3 before and I want to assert that a file is uploaded or at the very least the upload method for S3 is called.

So, all I'm attempting to do is unit test a method that uploads a file to an S3 container. Here is the test I have written so far to handle this.

Note: I am using NSubstitue for mocking in my tests.

using Amazon.S3;
using Amazon.S3.Model;
using Amazon.S3.Transfer;
using AmazonStorage.Repositories;
using Microsoft.VisualStudio.TestTools.UnitTesting;
using NSubstitute;
using System;
using System.IO;
using System.Threading;
using System.Threading.Tasks;

namespace AmazonStorage.Tests.Repositories
{
    [TestClass]
    public class FileRepositoryTests
    {
        private readonly FileRepository _sut;
        private readonly IAmazonS3 _client = Substitute.For<IAmazonS3>();
        private readonly TransferUtility _transfer = Substitute.For<TransferUtility>();
   
    public FileRepositoryTests()
    {
        var request = new GetObjectRequest()
        {
            Key = "somefile.txt",
            BucketName = "aws-s3-apadmi"
        };
        _client.GetObjectAsync(request);
        _sut = new FileRepository(_client);
    }

    [TestMethod]        
    public async Task PutFileInStorage_ShouldReceieveCallToUploadAsyc()
    {   
        // Arrange
        await _transfer.UploadAsync(Arg.Any<string>(), Arg.Any<string>(), Arg.Any<CancellationToken>());
       
        //Assert
        // I was thinking of something like this?
        // await _client.Received(1).UploadObjectFromStreamAsync(Arg.Any<string>(), Arg.Any<string>(), Arg.Any<Stream>());
    }

As you'll see I define my system under test, I bring in and mock the IAmazonS3 interface, I also mock the TransferUtility which is typically called when you upload a file to S3.

The FileRepository class has a file exists check which you'll see below in the method and that is why I pass the GetObjectAsync with a mocked GetObjectRequest into the FileRepository .

Here is my method to upload a file to S3.

private readonly IAmazonS3 _client;

public FileRepository(IAmazonS3 client)
{
    _client = client;
}

public async Task PutFileInStorage(string fileName, string content)
{      
    try
    {
        await UploadToS3(fileName, content);
    }
    catch (Exception ex)
    {
        throw new Exception(ex.Message);
    }
}
 
public async Task UploadToS3(string fileName, string content)
{            
    using MemoryStream ms = new(Encoding.UTF8.GetBytes(content));
         
    var uploadRequest = new TransferUtilityUploadRequest
    {
        InputStream = ms,
        Key = fileName,
        BucketName = "aws-s3-apadmi"                    
    };
    
    bool fileCheck = await CheckFileExists(uploadRequest.Key, uploadRequest.BucketName);
    
    if (fileCheck.Equals(true))
            throw new Exception("File already exists");                

    if (!fileCheck)
    {
        // Upload file to storage
        try
        {
            var fileTransferUtility = new TransferUtility(_client);
            await fileTransferUtility.UploadAsync(uploadRequest);
        }
        catch (Exception ex)
        {
            throw new Exception(ex.Message);
        }
    }
}

The question is, how do I assert that this file has uploaded or at the very least that the UploadAsync has been called once?

Answer 1

Tight coupling to 3rd party dependencies (ie TransferUtility ) is the design issue here

//...

var fileTransferUtility = new TransferUtility(_client);
 
//...

This makes testing your code in isolation difficult.

Consider adding an additional layer of abstraction that wraps TransferUtility to give you more control over your code.

public interface ITransferUtility {
    Task UploadAsync(TransferUtilityUploadRequest request);
}

public class FileTransferUtility : ITransferUtility {
    private readonly IAmazonS3 client;
    
    public FileTransferUtility (IAmazonS3 client) {
        this.client = client;
    }
    
    public async Task UploadAsync(TransferUtilityUploadRequest uploadRequest) {
        bool fileCheck = await CheckFileExists(uploadRequest.Key, uploadRequest.BucketName);

        if (fileCheck)
            throw new Exception("File already exists");
        
        // Upload file to storage
        try {
            TransferUtility fileTransferUtility = new TransferUtility(client);
            await fileTransferUtility.UploadAsync(uploadRequest);
        }
        catch (Exception ex) {
            throw new Exception(ex.Message);
        }
        
    }
    
    //...CheckFileExists and other necessary members
}

Also note the Separation of Concerns / Single Responsibility Principle (SoC / SRP) applied by refactoring most of the checks done in the repository to the new utility

This now simplifies the repository to focus on its core concern

private readonly ITransferUtility fileTransferUtility;

public FileRepository(ITransferUtility fileTransferUtility) {
    this.fileTransferUtility = fileTransferUtility;
}

public async Task PutFileInStorage(string fileName, string content) {
    try {
        await UploadToS3(fileName, content);
    } catch (Exception ex) {
        throw new Exception(ex.Message);
    }
}
 
private async Task UploadToS3(string fileName, string content)
    using MemoryStream ms = new(Encoding.UTF8.GetBytes(content));
         
    TransferUtilityUploadRequest uploadRequest = new TransferUtilityUploadRequest {
        InputStream = ms,
        Key = fileName,
        BucketName = "aws-s3-apadmi"
    };
    
    await fileTransferUtility.UploadAsync(uploadRequest);
}

The subject under test is now able to be isolated to verify expected behavior

[TestClass]
public class FileRepositoryTests {
    private readonly FileRepository sut;
    private readonly ITransferUtility transfer = Substitute.For<ITransferUtility>();

    public FileRepositoryTests() {
        sut = new FileRepository(transfer);
    }

    [TestMethod]
    public async Task PutFileInStorage_ShouldReceieveCallToUploadAsyc()
        //Arrange
        transfer.UploadAsync(Arg.Any<TransferUtilityUploadRequest>()).Returns(Task.CompletedTask);
        
        string fileName = "somefile.txt";
        string bucketName = "aws-s3-apadmi";
        string content = "Hello World";
        
        //Act
        await sut.PutFileInStorage(fileName, content);
       
        //Assert
        transfer.Received().UploadAsync(Arg.Is<TransferUtilityUploadRequest>(x =>
            x.Key == fileName 
            && x.BucketName = bucketName
        ));
    }
}