ASP.NET 核心手动处理请求体 stream

Question

I have a method for uploading a CSV file to server. If this CSV file has 100 000 rows, then processing takes 10 minutes. So I tried to return a result after 2 seconds and continue processing in background task:

[HttpPost]
[RequestSizeLimit(53 * 1024 * 1024)]
public async Task<IActionResult> Load(IFormFile data)
{
    await await Task.WhenAny(
            Task.Delay(TimeSpan.FromSeconds(2)),
            AsyncProcessData(data));
    return RedirectToAction("Index");
}

In case of executing takes more than 2 seconds, user will see that last upload is in status "Processing", and I decided it is better, than if user has unresponding web page for several minutes.

Unfortunatly, result of this approach is that only 1760 records were processed and while reading 1761-th row an error was throwed - Cannot access a closed file.

As far as I can understand, after 2 seconds completed response was sended and stream with incoming data was disposed.

I would like ASP.NET infrastructure to don't dispose stream with request body so I could do it myself, something like this:

Task.WhenAny(
    Task.Delay(TimeSpan.FromSeconds(2)),
    AsyncProcessData(data).ContinueWith(_ => <dispose_stream_with_request_body>));

Can I do it?

Answer 1

This is not possible.

Think of it this way: if you return a response ( return RedirectToAction("Index"); ) then the client (the browser?) will stop sending data and therefore ending the stream even if you somehow manage to avoid disposing it server-side. If the client stops sending data there's nothing you can do server-side.

A couple of suggestions for what you can do...

Load all data into memory

This is probably the closest you can get to your current workflow.

I'm not sure what happens in AsyncProcessData but I imagine you are line-by-line/chunk-by-chunk reading data from the request stream and doing some processing on it one at a time. If you instead write the whole stream into another stream or similar and then use that new stream instead for your processing, you can schedule the processing on a background thread and return a response as soon as the stream is copied.

Note : this means you are keeping the whole payload from the request in-memory, which does not scale very well. In other words, if you allow uploading large files or many users are using it at the same time, you'll likely run out of memory quickly.

Saving to disk/DB and processing asynchronously

I would suggest something like the following instead.

When Load is called you save the payload (the CSV-file) to a temporary file on your disk or in a database. Then you schedule the processing of this file on a background thread and return a response like you do now. Here you probably need to delete the temporary file (from disk or in the database) when you're done.

Note : like the memory issue with the other solution here you need to ensure you have the necessary disk space. Disk space is usually a lot cheaper than memory, so this is much more scalable. You do, however, need to ensure you remember to delete the file when you're done (also if something goes wrong and your app crashes).