我应该选择 IAsyncEnumerable<T> 或 IEnumerable <Task<T> &gt;?

Question

in my C# program, I have a list that has 100+ URLs and I will download files from those URLs.

at first, I will use list.select(x=> DosomethingAsync(x)) then use Task.WhenAny to handle the result (these step is very fast, the main cost is file download)

while (!itemsTasks.IsNullOrEmpty())
{
    Task<Stream> fininshed_item_task = await Task.WhenAny(itemsTasks);
    Stream itemResult = await fininshed_item_task;
            
    // do something with   itemResult
    ......
    itemsTasks.Remove(fininshed_item_task);
}

now I hear that there is IAsyncEnumerable and it seems could keep my code more straightforward.

but I am very worried IAsyncEnumerable will fetch file one by one and cost more time. And I find that IAsyncEnumerable do a lot of thread changing!

IEnumerable<Task<T>> seems to start tasks immediately before await. and another problem is I need to find a way to cancel the remaining tasks if there is one get failed.

Can you explain this?

Answer 1

Basically IAsyncEnumerable is an enumeration of awaited tasks' result while IEnumerable<Task> is an enumeration of tasks.

I would say that IAsynEnumerable is a shortcut feature for a specific usecase of IEnumerable<Task<T>> .

Let's say we have this expensive operation to perform

static async Task<int> ExpensiveOperationAsync(int number)
{
    // some expensive operation
    await Task.Delay(100);

    return number;
}

To return the operation's result on all of the positives numbers you obviously could not just return a list of values since there is an infinite amount of them, so you would have to yield every result once the task has completed.

But this does not compile since int is not awaitable and the method is async .

static async int GetNumbersAsync()
{
    var i = 0;
    
    while (true)
    {
        yield return await ExpensiveOperationAsync(i++);
    }
}

One way around is to not use an aync method and just return the tasks, here we have our IEnumerable<Task<int>> .

static IEnumerable<Task<int>> GetNumbersAsyncEnumerableTasks()
{
    var i = 0;
    
    while (true)
    {
        yield return ExpensiveOperationAsync(i++);
    }
}

And use it

foreach (var task in GetNumbersAsyncEnumerableTasks())
    Console.WriteLine(await task);

But one downside of this approach is that it's not very explicit about what you are trying to achieve.

Instead C# provides a nicer way to do this, the IAsyncEnumerable<T> which is a kind of an awaitable thing.

static async IAsyncEnumerable<int> GetNumbersAsyncEnumerable(int max)
{
    for (var i = 0; i < max + 1; i++)
    {
        yield return await ExpensiveOperationAsync(i);
    }
}

// notice the await foreach syntax
await foreach (var number in GetNumbersAsyncEnumerable(10))
    Console.WriteLine(number);

As you can see IAsynEnumerable<T> is useful for a few specific situations but it does not seem to fit your case.

I rather do something like this

public class MyProcessor
{
    public async Task ProcessAllTasksAsync<T>(IEnumerable<Task<T>> tasks)
    {
        var wrappedTasks = tasks.Select(TaskWrapperAsync);
        await Task.WhenAll(wrappedTasks);
    }

    private async Task TaskWrapperAsync<T>(Task<T> task)
    {
        var result = await task;
        DoSomethingWithItemResult(result);
    }

    private void DoSomethingWithItemResult<T>(T result)
    {
        // some cool stuff
    }
}