Fastest Method for Downloading Large Azure Storage Blobs?

Question

I'm working on a project in C# where I need to quickly download potentially large blobs from azure storage to a local file using the Azure.Storage.Blobs SDK v12. However, the times I'm getting when trying to download a Blob that's about 2.8 GB in size is a bit slow for my needs.

I'm using the DownloadToAsync method that I've seen suggested elsewhere:

Response response = await blob.DownloadToAsync(
   destinationPath,
   blobDownloadToOptions,
   cancellationToken).ConfigureAwait(false);

The blobDownloadTo options contains my TransferOptions, which are basically:

MaximumConcurrency = 8
InitialTransferSize = blobSize / 8
MaximumTransferSize = blobSize / 8

And also my progress handler which let's me know how fast the download is going. When I run it locally using Azure Emulated Storage, I can see via Fiddler than the request is happening across the desired number of threads and it only takes around 6-7 seconds. But once it's deployed to my service, the request suddenly takes ~30 seconds.

Download Progress: (356725906) in : (4361) ms.
Download Progress: (713461892) in : (8704) ms.
Download Progress: (1070184646) in : (11687) ms.
Download Progress: (1426907400) in : (16048) ms.
Download Progress: (1783630154) in : (19321) ms.
Download Progress: (2140352908) in : (23769) ms.
Download Progress: (2497075662) in : (26981) ms.
Download Progress: (2853782037) in : (30178) ms.

Are there any tips or recommendations people have that'll help me up my perf? I'm unable to utilize the DataMovement Library as it's not compatible with v12.

Answer 1

• I would suggest you to please use the Azure speed test tool to test the speed of the blob data being downloaded . The Azure Speed test tool download link is given below: -

https://www.azurespeed.com/Azure/Download

Also, would suggest you to please use the 'AzCopy' tool to check whether you are getting the same download speeds through it as it would clear your doubts about any network drops .

• You can also check the below link for Azure storage resource performance and the scalability checklist which can surely help you to enhance the downloading speed of the blobs : -

https://docs.microsoft.com/en-us/azure/storage/blobs/storage-performance-checklist?toc=%2Fazure%2Fstorage%2Fblobs%2Ftoc.json#subheading2

Finally, you can try to use the following code for downloading blob from the Azure storage as well as suggest you use the 'StorageTransferOptions' class in the.Net Azure SDK code in which you can specify the 'MaximumConcurrency' and 'MaximumTransferSize' parameters and variables accordingly to maximize the download speed of the blobs from the Azure storage .

var ms = new MemoryStream();
await blobClient.DownloadToAsync(ms).ConfigureAwait(false);
var blobStream = await blobClient.OpenReadAsync().ConfigureAwait(false);
var props = await blobClient.GetPropertiesAsync().ConfigureAwait(false);
return File(blobStream, props.Value.ContentType, blobClient.Name);

For more detailed information, you can surely refer the below community thread link and the link below for more clarification: -

Downloading blob results in more memory being used than size of actual blob

https://docs.microsoft.com/en-us/answers/questions/89440/getting-very-slow-speed-while-downloading-storage.html