如何在不使用磁盘且内存不足的情况下将大型文件从api流式传输到api？

Question

I have a API HTTPGET that retrieves a file from another API, and spits it out. This works for smaller files, but the problem I'm having is that the files retrieved can be rather large in size (up to 2gb), and the MemoryStream is a limitation. Any ideas how to stream the file content without using disk and avoiding the 'out of memory' exception?

Controller:

[Route("{id}/file", Name = "GetContentFile")]
[HttpGet]
public IHttpActionResult GetContentFile(string id)
{
    if (String.IsNullOrEmpty(id))
        return BadRequest();
    ContentFile cfl = new ContentFile();
    var ret = new HttpResponseMessage(HttpStatusCode.OK);
    try
    {
        cfl = otcrepo.GetContentFile(id);
        var mstream = new MemoryStream(cfl.Data);
        ret.Content = new StreamContent(mstream);
        ret.Content.Headers.ContentType = new MediaTypeHeaderValue(cfl.ContentType);
        ret.Content.Headers.ContentDisposition = new ContentDispositionHeaderValue("attachment");
    }
    catch
    {
        return InternalServerError();
    }
    if (cfl != null)
    {
        ResponseMessageResult responseMessageResult = ResponseMessage(ret);
        return responseMessageResult;
    }
    else
    {
        return NotFound();
    }
}

Model:

public class ContentFile
{
    public string Filename { get; set; }
    public byte[] Data { get; set; }
    public StreamContent DataStream { get; set; }
    public string ContentType { get; set; }
}

Repository call:

public ContentFile GetContentFile(string id)
{
    ContentFile fl = new ContentFile();
    using (var htc = new HttpClient())
    {
        var response = htc.GetAsync(ConfigurationManager.AppSettings["BaseUrl"] + "/api/v2/nodes/" + id + "/content/").Result;

        fl.Data = response.Content.ReadAsByteArrayAsync().Result;
        fl.ContentType = response.Content.Headers.GetValues("Content-Type").FirstOrDefault();
    }
    return fl;
}

Thanks.

Answer 1

So instead of GetContentFile reading the complete stream into an array and returning it we return the stream. Than there is no need for the MemoryStream .

[Route("{id}/file", Name = "GetContentFile")]
[HttpGet]
public async Task<IHttpActionResult> GetContentFile(string id)
{
    ...
    var result = await otcrepo.GetContentFile(id);
    ret.Content = new StreamContent(result.stream);
    ret.Content.Headers.ContentType = new MediaTypeHeaderValue(result.contentType);
    ...

}

public async Task<(Stream stream, string contentType)> GetContentFile(string id)
{
    var htc = new HttpClient()
    var response = await htc.GetAsync(ConfigurationManager.AppSettings["BaseUrl"] + "/api/v2/nodes/" + id + "/content/");

    var stream = await response.Content.ReadAsStreamAsync();
    var contentType = response.Content.Headers.GetValues("Content-Type").FirstOrDefault();
    return (stream, contentType);
}