C# AsyncEnumerable running/awaiting multiple tasks never finishes

Question

I want to have a function which receives a Task<bool> and running it in X tasks.

For that I've wrote the below code:

public static class RetryComponent
{
    public static async Task RunTasks(Func<Task<bool>> action, int tasks, int retries, string method)
    {
        // Running everything
        var tasksPool = Enumerable.Range(0, tasks).Select(i => DoWithRetries(action, retries, method)).ToArray();
        await Task.WhenAll(tasksPool);
    }

    private static async Task<bool> DoWithRetries(Func<Task<bool>> action, int retryCount, string method)
    {
        while (true)
        {
            if (retryCount <= 0)
                return false;

            try
            {
                bool res = await action();
                if (res)
                    return true;
            }
            catch (Exception e)
            {
                // Log it
            }

            retryCount--;
            await Task.Delay(200); // retry in 200
        }
    }
}

And the following execution code:

BlockingCollection<int> ints = new BlockingCollection<int>();
foreach (int i in Enumerable.Range(0, 100000))
{
    ints.Add(i);
}
ints.CompleteAdding();

int taskId = 0;
var enumerable = new AsyncEnumerable<int>(async yield =>
{
    await RetryComponent.RunTasks(async () =>
    {
        try
        {
            int myTaskId = Interlocked.Increment(ref taskId);

            // usually there are async/await operations inside the while loop, this is just an example

            while (!ints.IsCompleted)
            {
                int number = ints.Take();

                Console.WriteLine($"Task {myTaskId}: {number}");
                await yield.ReturnAsync(number);
            }
        }
        catch (InvalidOperationException)
        {
            return true;
        }
        catch (Exception e)
        {
            Console.WriteLine(e);
            throw;
        }

        return true;
    }, 10, 1, MethodBase.GetCurrentMethod().Name);
});

await enumerable.ForEachAsync(number =>
{
    Console.WriteLine(number);
});

where AsyncEnumerable is from System.Collections.Async .

The console shows Task 10: X (where x is a number in the list..).

When I remove the AsyncEnumerable everything works as intended (all tasks are printing and the execution ends).. For some reason, which I cannot find for a lot of time, using AsyncEnumerable just ruins everything (In my main code, I need it to use AsyncEnumerable .. scalability stuff..) meaning that code never stops and only the last task (10) is printing. when i added more logs, i see that tasks 1-9 never finish.

So just to clear things up, I want to have multiple tasks doing async operations and yield the results to a single AsyncEnumerable object which acts as a pipe. (this was the idea..)

Answer 1

The problem is that the enumerator/generator pattern is sequential, but you're trying to do a multi-producer, single consumer pattern. Since you use nested anonymous functions, and stack overflow doesn't show line numbers, it's hard to describe exactly which part of the code I'm referring to, but I'll try anyway.

The way that AsyncEnumerable works is basically to wait for the producer to produce a value, then wait for the consumer to use the value, then repeat. It does not support the producer and consumer running at different speeds, hence why I say this pattern is sequential. It does not have a queue of produced items, only the current value . ReturnAsync does not wait for the consumer to use the value, instead you are supposed to await the task that it returns, which gives you a signal that it's ready. Therefore we can conclude that it's not thread-safe.

However, RetryComponent.RunTasks runs 10 tasks in parallel and that code calls yield.ReturnAsync without checking if anyone else has already called it and if so if that task is complete. Since the Yield class only stores the current value, your 10 concurrent tasks overwrite the current value without waiting for the Yield object to be ready for a new value, so 9 of the tasks are lost and are never awaited. Since those 9 tasks are never awaited, the methods never complete and Task.WhenAll never returns, and neither do any of the other methods in the entire call stack.

I created an issue on github proposing they improve their library to throw exceptions when this happens. If they implement it, your catch block would write the message to the console and rethrow the error, putting the task in a faulted state, which would allow task.WhenAll to complete and therefore your program wouldn't have hung.

You could use multi-threaded synchronization APIs to ensure only one task at a time calls yield.ReturnAsync and await the return task. Or you could avoid using a multi-producer pattern as a single producer can be an enumerator easily. Otherwise you'll need to completely rethink how you want to implement the multi-producer pattern. I suggest TPL Dataflow which is built-in to .NET Core and available in the .NET Framework as a NuGet package.

Answer 2

@zivkan is absolutely right about the sequential producer pattern. If you want to have concurrent producers that for a single stream, it is still possible to implement with the AsyncEnumerable library, but requires some extra code.

Here is an example of a solution for the problem with concurrent producers and consumers (only one consumer in this case):

        static void Main(string[] args)
        {
            var e = new AsyncEnumerable<int>(async yield =>
            {
                var threadCount = 10;
                var maxItemsOnQueue = 20;

                var queue = new ConcurrentQueue<int>();
                var consumerLimiter = new SemaphoreSlim(initialCount: 0, maxCount: maxItemsOnQueue + 1);
                var produceLimiter = new SemaphoreSlim(initialCount: maxItemsOnQueue, maxCount: maxItemsOnQueue);

                // Kick off producers
                var producerTasks = Enumerable.Range(0, threadCount)
                    .Select(index => Task.Run(() => ProduceAsync(queue, produceLimiter, consumerLimiter)));

                // When production ends, send a termination signal to the consumer.
                var endOfProductionTask = Task.WhenAll(producerTasks).ContinueWith(_ => consumerLimiter.Release());

                // The consumer loop.
                while (true)
                {
                    // Wait for an item to be produced, or a signal for the end of production.
                    await consumerLimiter.WaitAsync();

                    // Get a produced item.
                    if (queue.TryDequeue(out var item))
                    {
                        // Tell producers that they can keep producing.
                        produceLimiter.Release();
                        // Yield a produced item.
                        await yield.ReturnAsync(item);
                    }
                    else
                    {
                        // If the queue is empty, the production is over.
                        break;
                    }
                }
            });

            e.ForEachAsync((item, index) => Console.WriteLine($"{index + 1}: {item}")).Wait();
        }

        static async Task ProduceAsync(ConcurrentQueue<int> queue, SemaphoreSlim produceLimiter, SemaphoreSlim consumerLimiter)
        {
            var rnd = new Random();
            for (var i = 0; i < 10; i++)
            {
                await Task.Delay(10);
                var value = rnd.Next();

                await produceLimiter.WaitAsync(); // Wait for the next production slot
                queue.Enqueue(value); // Produce item on the queue
                consumerLimiter.Release(); // Notify the consumer
            }
        }