Long Running Task in .Net Core

Question

I am currently working on WebApi using.Net Core, one of my Api Method will call number of another Api (3rd party), and it will take some time to return response, but I don't want our Api consumers to wait for response, instead I want to return early response ie The Operation is started. And I ll provide an endpoint to our Consumers through which they can get the status of that operation. For example our consumer calls the api to generate 100k records for which my Api will call around 20 parallel calls to 3rd party api. So I don't want consumer for these 20 apis response. Currently I have this code:

 public async Task<ActionResult> GenerateVouchers([FromBody][Required]CreateVoucherRequestModel request, string clientId)
    {
        _logger.LogInformation(Request.Method, Request.Path);
        // await _voucherService.ValidateIdempotedKeyWithStatus(clientId, _idempotentHeader);
        //TODO: Check Voucher type & Status before Generating Voucher
        var watch = Stopwatch.StartNew();
        var vouchers = new List<VoucherCreateResponseModel>();
        var batchSize = 5000;
        int numberOfBatches = (int)Math.Ceiling((double)request.quantity / batchSize);
        int totalVoucherQuantity = request.quantity;
        request.quantity = 5000;
        var tasks = new List<Task<VoucherCreateResponseModel>>();
        for (int i = 0; i < numberOfBatches; i++)
        {
            tasks.Add(_client.GenerateVoucher($"CouponsCreate", request));
            vouchers.AddRange(await Task.WhenAll(tasks).ConfigureAwait(false));
        }

        // await _voucherService.GenerateVouchers(request, clientId, _idempotentHeader);
        watch.Stop();
        var totalMS = watch.ElapsedMilliseconds;

        return Ok();
    }

But the issue with above code even though I have ConfigureAwait(false) , it waits for all 20 requests to execute and when response of all requests are returned than api consumer will get response, but each each of these 20 request will take around 5 seconds to execute, so our consumers may get request timeout while waiting for response.

How can I fix such issue in.Net Core.

Answer 1

It's not a good practice to wait for long running process inside controller. In my project how I handle this is,

1. put the data necessary (something like a Id for a batch) for long running process to a Azure queue within the API
2. trigger a function app from the particular queue, So API's responsibility is putting the data in to the queue
3. From there on it's function apps responsibility to complete process.
4. May be using something like signalR you can notify the frontend when process is completed