Cannot upload a file to Azure Storage C#?

Question

I am trying a simple upload to Azure Storage in C#. It works for small files, but when I try to upload a file of 40mb, it just wont upload. I am stuck in upload for about 20 minutes and then I end up cancelling the call.

var storageConnectionString = "XXXXXXX";
            var storageAccount = CloudStorageAccount.Parse(storageConnectionString);
            var blobClient = storageAccount.CreateCloudBlobClient();

            //Gets the folder with the MediaID
            CloudBlobContainer videoscontainer = blobClient.GetContainerReference("TestFolder");

            CloudBlockBlob videofile = videoscontainer.GetBlockBlobReference(("test" + ".mp4"));


            //Save blob contents to a temp video file.
            var videostream = new FileStream(@"XXXXXX.mp4", FileMode.Open, FileAccess.Read);
            videofile.UploadFromStream(videostream);
            videostream.Close();

I have a decent internet so I am sure an upload of just 40mb cant takes more than 20 minutes to upload. The hardest part is, I don't know what error I am getting. I tried a manual upload in the storage and it finished in about 2 minutes.

EDIT: I just kept waiting until i got a callback and got this error.

Error:

"Message": "An error has occurred.",
    "ExceptionMessage": "The request was aborted: The request was canceled.",
    "ExceptionType": "Microsoft.WindowsAzure.Storage.StorageException",
    "StackTrace": "   at Microsoft.WindowsAzure.Storage.Core.Executor.Executor.ExecuteSync[T](RESTCommand`1 cmd, IRetryPolicy policy, OperationContext operationContext) in c:\\Program Files (x86)\\Jenkins\\workspace\\release_dotnet_master\\Lib\\ClassLibraryCommon\\Core\\Executor\\Executor.cs:line 604\r\n   at Microsoft.WindowsAzure.Storage.Blob.CloudBlockBlob.UploadFromStreamHelper(Stream source, Nullable`1 length, AccessCondition accessCondition, BlobRequestOptions options, OperationContext operationContext) in c:\\Program Files (x86)\\Jenkins\\workspace\\release_dotnet_master\\Lib\\ClassLibraryCommon\\Blob\\CloudBlockBlob.cs:line 411\r\n   at Microsoft.WindowsAzure.Storage.Blob.CloudBlockBlob.UploadFromStream(Stream source, AccessCondition accessCondition, BlobRequestOptions options, OperationContext operationContext) in c:\\Program Files (x86)\\Jenkins\\workspace\\release_dotnet_master\\Lib\\ClassLibraryCommon\\Blob\\CloudBlockBlob.cs:line 309\r\n   at StoneFunctions.Controllers.ValuesController.<UploadSingleFile>d__0.MoveNext() in C:\\Users\\Alan\\source\\repos\\Stone\\StoneFunctions\\Controllers\\ValuesController.cs:line 49\r\n--- End of stack trace from previous location where exception was thrown ---\r\n   at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)\r\n   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)\r\n   at System.Threading.Tasks.TaskHelpersExtensions.<CastToObject>d__3`1.MoveNext()\r\n--- End of stack trace from previous location where exception was thrown ---\r\n   at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)\r\n   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)\r\n   at System.Web.Http.Controllers.ApiControllerActionInvoker.<InvokeActionAsyncCore>d__0.MoveNext()\r\n--- End of stack trace from previous location where exception was thrown ---\r\n   at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)\r\n   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)\r\n   at System.Web.Http.Controllers.ActionFilterResult.<ExecuteAsync>d__2.MoveNext()\r\n--- End of stack trace from previous location where exception was thrown ---\r\n   at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)\r\n   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)\r\n   at System.Web.Http.Dispatcher.HttpControllerDispatcher.<SendAsync>d__1.MoveNext()",
    "InnerException": {
        "Message": "An error has occurred.",
        "ExceptionMessage": "The request was aborted: The request was canceled.",
        "ExceptionType": "System.Net.WebException",
        "StackTrace": "   at System.Net.ConnectStream.InternalWrite(Boolean async, Byte[] buffer, Int32 offset, Int32 size, AsyncCallback callback, Object state)\r\n   at System.Net.ConnectStream.Write(Byte[] buffer, Int32 offset, Int32 size)\r\n   at Microsoft.WindowsAzure.Storage.Core.ByteCountingStream.Write(Byte[] buffer, Int32 offset, Int32 count) in c:\\Program Files (x86)\\Jenkins\\workspace\\release_dotnet_master\\Lib\\ClassLibraryCommon\\Core\\ByteCountingStream.cs:line 180\r\n   at Microsoft.WindowsAzure.Storage.Core.Util.StreamExtensions.WriteToSync[T](Stream stream, Stream toStream, Nullable`1 copyLength, Nullable`1 maxLength, Boolean calculateMd5, Boolean syncRead, ExecutionState`1 executionState, StreamDescriptor streamCopyState) in c:\\Program Files (x86)\\Jenkins\\workspace\\release_dotnet_master\\Lib\\Common\\Core\\Util\\StreamExtensions.cs:line 169\r\n   at Microsoft.WindowsAzure.Storage.Core.Executor.Executor.ExecuteSync[T](RESTCommand`1 cmd, IRetryPolicy policy, OperationContext operationContext) in c:\\Program Files (x86)\\Jenkins\\workspace\\release_dotnet_master\\Lib\\ClassLibraryCommon\\Core\\Executor\\Executor.cs:line 670"
    }

Answer 1

I believe the error is coming because of BlobRequestOptions.SingleBlobUploadThresholdInBytes value. This value essentially specifies the maximum size of the blob that can be uploaded without breaking it into chunks. If you don't specify any value for this property, the default value is 128MB . Since you're not setting this explicitly and your blob size is 40MB , Azure Storage SDK is trying to upload this 40MB data in a single request without breaking it into chunks and that request is somehow timing out.

To solve this problem, please explicitly set this value to say 2MB or 4MB . Then the SDK will break your 40MB file in 20 (or 10) chunks and upload these chunks.

Here's the code I used:

        var cred = new StorageCredentials(accountName, accountKey);
        var account = new CloudStorageAccount(cred, true);
        CloudBlobClient client = account.CreateCloudBlobClient();
        client.DefaultRequestOptions = new BlobRequestOptions()
        {
            SingleBlobUploadThresholdInBytes = 2 * 1024 * 1024
        };
        var container = client.GetContainerReference("myblobcontainer");
        container.CreateIfNotExists();
        var filePath = @"Full File Path";
        var blob = container.GetBlockBlobReference("blob name.exe");
        using (var fs = new FileStream(filePath, FileMode.Open))
        {
            blob.UploadFromStream(fs);
        }

I uploaded about 60MB file using this code.