在.net Core中使用HttpClient下载分块编码的文件

Question

I'm using an API to download a file using HttpClient. When my entire file fits in a single request, I'm able to save the file and open it correctly.

When the entire file does not fit in my maximum chunk size, I have to do multiple requests to my API to get the file and it's returned chunked encoded. By guess is that the chunks are not decoded properly and end up in my file which corrupts it.

Is there a way to chunk decode the response from GetStreamAsync?

public byte[] GetFileContent(File file)
{
    var baseFileUri = BASE_URI + $"platform/files/{file.Id}";

    int chunkSize = 128 * 1024;

    chunkSize = file.NativeSize <= chunkSize ? file.NativeSize : chunkSize;
    int start = 0;

    using (MemoryStream stream = new MemoryStream())
    {
        while (start < file.NativeSize)
        {
            Get(baseFileUri, new List<KeyValuePair<string, string>>() { new KeyValuePair<string, string>("offset", start.ToString()), new KeyValuePair<string, string>("size", chunkSize.ToString()) }, stream);

            start = start + chunkSize;

            if (file.NativeSize < start + chunkSize)
            {
                chunkSize = file.NativeSize - start;
            }
        }

        return stream.ToArray();
    }
}

private void Get(string uri, List<KeyValuePair<string, string>> parameters, Stream stream)
{
    var requestUri = BuildUriString(uri, parameters);

    var methodResult = _client.GetStreamAsync(requestUri);
    methodResult.Result.CopyTo(stream);
}

Answer 1

In your code you are getting the chunks manually by doing multiple request. When the requested API supports chunked responses you could probably try to set the "Transfer-Encoding": "chunked" header and let HttpClient library handle the chunked response. The header could be set on the HttpClient as following:

_client.DefaultRequestHeaders.TransferEncodingChunked = false;