Web服务从Azure下载文件引发大文件异常

Question

I'm working on a web service framework in Azure, as well as a website to communicate with Azure. The website should make a call to Azure with a filename, and the web service in Azure should find the given file in blob storage and send it back to the client for downloading.

I can get this to work as long as the file being returned is small (testing with a 20mb mp4 file works fine), but something like a 1Gb mp4 file ends up throwing an exception.

C# request code:

public override async Task ProcessRequestAsync(HttpContext context)
{
    JavaScriptSerializer oSearlizer = new JavaScriptSerializer();
    object o = new
    {
        sourceStorageAccountName = "accountName",
        sourceStorageAccountKey = "accountKey",
        sourceContainer = "test"
    };

    string req = oSearlizer.Serialize(o);
    HttpContent content = new StringContent(req, Encoding.UTF8, "application/json");
    HttpClient client = new HttpClient();

    HttpResponseMessage x = await client.PostAsync("http://localhost:7071/api/get_file", content);
    context.Response.Clear();
    context.Response.Buffer = false;
    context.Response.BufferOutput = false;
    context.Response.ContentType = "video/mp4";
    context.Response.AddHeader("content-disposition", "attachment;filename=test.mp4"); // Save file         
    context.Response.Charset = "";
    context.Response.Cache.SetCacheability(HttpCacheability.NoCache);
    context.Response.BinaryWrite(await x.Content.ReadAsByteArrayAsync());
    context.Response.End();
}

Azure webservice code:

CloudBlobContainer sourceBlobContainer = CopyBlobHelpers.GetCloudBlobContainer(_sourceStorageAccountName, _sourceStorageAccountKey, _sourceContainer);
CloudBlockBlob blob = sourceBlobContainer.ListBlobs().First();

HttpResponseMessage response = new HttpResponseMessage(HttpStatusCode.OK);
Stream stream = blob.OpenRead(null, new BlobRequestOptions() { ServerTimeout = new System.TimeSpan(2, 59, 59), MaximumExecutionTime = new System.TimeSpan(2, 59, 59) });
response.Content = new StreamContent(stream);
response.Content.Headers.ContentDisposition = new ContentDispositionHeaderValue("attachment");
response.Content.Headers.ContentDisposition.FileName = fileNameZip;
response.Content.Headers.ContentType = new MediaTypeHeaderValue("video/mp4");

return response;

Like I said, this code works fine for an mp4 file of 20mb, but throws an exception for a 1Gb mp4 file or larger. The exception I get is: "Exception: SocketException: An existing connection was forcibly closed by the remote host"

If I remove the "await"'s from the request code and use ".Result" instead, I get a different exception: "Exception: HttpRequestException: Error while copying content to a stream."

Stepping through the code, it enters "await client.PostAsync()", I step through the Azure web service code until the return statement. Then I step again and nothing happens for 1-2min, and then the exception is thrown.

Does anyone have any ideas about what might be going on? I'm thinking it has something to do with the stream trying to push through too much data or a time-out somewhere, but nothing I'm doing seems to be solving the issue.

Answer 1

I added and then deleted a comment about using HttpClient.Timeout because I think that, while that might fix the problem, there is an issue with the design you are using. You are using a POST request to get a resource. You really should be using a GET request. Normally, the way this works is that you would call something like

GET http://localhost:7071/api/{file_name}

where {file_name} is the name of the file that you want to download. The service would generally know the information about the storage account, and so you wouldn't need the request body that you currently have. It is possible that you want the service to be agnostic to which storage account to download from, but I think that's pretty unlikely.

I also feel like it's necessary to note that if you are going to continue sending the storage account information over the wire, you need to be using HTTPS instead of HTTP, because as it stands now, you are sending the storage account key in plaintext.

Answer 2

Due to there is a limit of default time out duration for Azure Functions, as below, so you will get the issue Exception: SocketException: An existing connection was forcibly closed by the remote host for downloading a large file from Azure Functions. Please refer to Functions limits .

Therefore, these small files can be downloaded in short time less than timeout duration, but the large file will be forcibly closed after timeout.

The solution is to response the blob url with SAS token from Azure Function to client, and then directly download the blob via sas url from Azure Blob Storage, because of the sas url without extra authentication and no timeout limits on Azure Storage.

If you are not familiar with how to generate SAS token for a blob, you can refer to the offical sample code Getting Started with Shared Access Signatures (SAS)