Downloading objects from Amazon S3 using AWS SDK - resultant file is corrupt

Question

I have a .Net Core 3.1 Web API which downloads an object (a PDF) from Amazon S3 to disk, using the AWS SDK library.

using Amazon.S3;
using Amazon.S3.Model;
using System.IO;

private void DownloadObject()
{
    BasicAWSCredentials awsCredentials = new Amazon.Runtime.BasicAWSCredentials("MyAccessKey", "MySecretKey");
    IAmazonS3 client = new Amazon.S3.AmazonS3Client(awsCreden‌​tials, Amazon.RegionEndpoint.USEast1);

    GetObjectRequest request = new GetObjectRequest
    {
        BucketName = "mybucket",
        Key = "test.pdf"
    };

    using (GetObjectResponse response = await client.GetObjectAsync(request))
    {
        using (Stream responseStream = response.ResponseStream)
        {
           using (StreamReader reader = new StreamReader(responseStream))
           {
               string responseBody = await reader.ReadToEndAsync();
               File.WriteAllText("C:\\test.pdf", responseBody);
           }
        }
    }
}

When the PDF downloads, the file size is wrong (too big) and if I open the PDF, all the pages are blank. This happens with other file types too. If I download a JPEG for example, I cannot open it - it's corrupt. Is it an encoding issue?

Answer 1

String encoding is not round-trippable for arbitrary binary data. That is to say, treating an abritrary byte[] array as UTF8, ASCII, etc. encoded text, converting byte -> string -> byte will often result in a different array of bytes than you started with. Presumably your PDF file contains binary data.

I recommend that you instead copy directly from one stream to another:

using (GetObjectResponse response = await client.GetObjectAsync(request))
{
    using (Stream responseStream = response.ResponseStream)
    using (FileStream outFile = File.Create("C:\\test.pdf"))
    {
        responseStream.CopyTo(outFile);
    }
}