Download 2 GB file in Service Fabric application

Question

I am designing an application which will support upload and download of XML file with limit of 2 GBs. I am using Azure Blob Storage as persistent storage to upload these files and then download them. Below piece of code throws OutOfMemory exception:

 var blob = this.GetBlockBlobReference(blobId, blobContainerName);
 return await blob.DownloadTextAsync().ConfigureAwait(false);

Also, I am not sure which approach will be faster and cleaner.

Answer 1

For faster downloading / uploading blobs, you can consider using Microsoft Azure Storage Data Movement Library . This library is designed for high-performance uploading, downloading and copying Azure Storage Blob and File.

You can set parallel threads via code below:

// Setup the number of the concurrent operations
TransferManager.Configurations.ParallelOperations = 64;

Then use TransferManager.UploadAsync or TransferManager.DownloadAsync method for uploading / downloading blobs respectively.

There is an example in github of uploading, you can modify it for downloading easily.