Stream data from an ASP.NET Core 3.1 endpoint without consuming huge amounts of memory

Question

Edit: I had a diagnostics middleware component that pulled out responses into a trace file. That was the culprit. So, if you find this because you have the same problem I had: check your middleware!

So at the call of a Web API endpoint, I'm gathering huge amounts of data (> 100 MB) from various data sources in a streaming asynchronous fashion. I want to forward that data to the client in a streaming fashion.

For this purpose I have built my own IActionResultExecutor<T> to generalize this as we have a few of these endpoints.

I have however noticed that the entire response is cached in memory before it's actually sent to the client. Not good. Obviously, I'm doing something wrong, but I cannot understand what I'm doing wrong!

The executor's ExecuteAsync looks like this:

public async Task ExecuteAsync(ActionContext context, AsyncStreamResult result)
{
   var bufferingFeature = context.HttpContext.Features.Get<IHttpResponseBodyFeature>();
   if (bufferingFeature != null)
      bufferingFeature.DisableBuffering();

   context.HttpContext.Response.StatusCode = StatusCodes.Status200OK;
   context.HttpContext.Response.ContentType = "application/json; charset=utf-8";

   var cancellationToken = context.HttpContext.RequestAborted;
   await context.HttpContext.Response.StartAsync(cancellationToken);
   await context.HttpContext.Response.WriteAsync("[", cancellationToken);
   bool seenFirstItem = false;

   await foreach (var item in result.Data) {
      if (seenFirstItem)
         await context.HttpContext.Response.WriteAsync(",", cancellationToken);

      await context.HttpContext.Response.BodyWriter.WriteAsync(item.JsonBytes, cancellationToken);

      seenFirstItem = true;
   }

   await context.HttpContext.Response.WriteAsync("]", cancellationToken);
}

I can see how the entire process allocates a lot of memory in a linear fashion in Visual Studio. I can also see curl not getting any data, until the request is finished. Then everything comes in one go. The irony is that curl reports that the data's transfer encoding is chunked . That's one big chunk, Some of the data streams are in excess of 100 MB. and I can't afford beefy pods in my k8s cluster for that reason alone, The way I see it, this operation should generate a bunch of Gen 0 objects. but nothing that the GC shouldn't be able to handle. Referenced objects should be counted in kilobytes!

I've tried sprinkle in some await context.HttpContext.Response.BodyWriter.FlushAsync(cancellationToken) , but nothing appears to make any difference.

What am I doing wrong?

Answer 1

Problem solved: I had some debug logging middleware in the pipeline that turned on buffering to trace responses sent to clients. DOH!