如何使用C＃HttpClient或AWS开发工具包通过CloudFront将视频上传到S3

Question

I'm trying to upload files (Images & Video) to an AWS CloudFront distribution, that points to an s3 bucket.

Currently, I can use the HttpClient to GET and PUT files using signed URLs generated via the CloudFront SDK.

using (HttpClient client = new HttpClient())
using (var stream = File.OpenRead(filepath))
using (HttpResponseMessage response = await client.PutAsync(url, new StreamContent(stream)))
{
    response.EnsureSuccessStatusCode();
}

I originally tried a POST, but this didn't work at all (it timed out after 30 seconds) and I found from this SO Answer that I need to add client.DefaultRequestHeaders.Add("x-amz-acl", "bucket-owner-full-control"); to give my object ACL access permissions so the bucket owner can access via the console.

I know I can upload to S3 using the AWS S3 SDK and I could enable transfer acceleration, though the AWS FAQ states that CloudFront is a better choice when uploading smaller files or datasets (< 1GB).

I've found the CloudFront documentation vague, wrong or non-existant for anything other than the initial setup of the CloudFront distribution.

Is the above method the correct way to upload any files to S3 via CloudFront, or is there an optimised, more robust way (eg multi-part uploads, so larger files can be resumed) - I want to optimise this for uploading Video, so if answers could focus on this it would be appreciated.

Answer 1

AWS Support suggest response in case this helps someone:

... three possible solutions. The first being that you can PUT objects to S3 via a signed URL to the S3 origin. The second option, PUTing the file through CF into S3 via S3 pre-signed URL. The third and most favorable option, using the Transfer Acceleration Endpoint to PUT the object into S3.

From my understanding, the FAQ stated that using CF instead of the TA endpoint is better for files smaller than 1 GB because TA is optimized for larger files. However, there are many factors that can influence the performance, I suggest testing both methods to see which service works best for your environment.

They also mention CF is much more complex to do multipart uploads:

Its going to be much more difficult if you need to use CloudFront signed URL for other reasons. You will need to use the Multipart Upload APIs (InitiateMultipartUpload, UploadPart, CompleteMultipartUpload) and sign them accordingly. Unfortunately we don't have any documentation or steps on how to do this. You can find more information on the Multipart Upload process here [2].

I highly recommend using the TransferUtility and S3 Transfer Acceleration endpoints if possible.