在Select linq查询中使用async / await

Question

After reading this post: Nesting await in Parallel.ForEach

I tried to do the following:

private static async void Solution3UsingLinq()
{
    var ids = new List<string>() { "1", "2", "3", "4", "5", "6", "7", "8", "9", "10" };

    var customerTasks = ids.Select(async i =>
    {
        ICustomerRepo repo = new CustomerRepo();
        var id = await repo.getCustomer(i);
        Console.WriteLine(id);

    });
}

For some reason, this doesn't work... I don't understand why, I think there is a deadlock but i'm not sure...

Answer 1

So at the end of your method, customerTasks contains an IEnumerable<Task> that has not been enumerated . None of the code within the Select even runs.

When creating tasks like this, it's probably safer to materialize your sequence immediately to mitigate the risk of double enumeration (and creating a second batch of tasks by accident). You can do this by calling ToList on your sequence.

So:

var customerTasks = ids.Select(async i =>
{
    ICustomerRepo repo = new CustomerRepo();
    var id = await repo.getCustomer(i); //consider changing to GetCustomerAsync
    Console.WriteLine(id);

}).ToList();

Now... what to do with your list of tasks? You need to wait for them all to complete...

You can do this with Task.WhenAll :

await Task.WhenAll(customerTasks);

You could take this a step further by actually returning a value from your async delegate in the Select statement, so you end up with an IEnumerable<Task<Customer>> .

Then you can use a different overload of Task.WhenAll :

IEnumerable<Task<Customer>> customerTasks = ids.Select(async i =>
{
    ICustomerRepo repo = new CustomerRepo();
    var c = await repo.getCustomer(i); //consider changing to GetCustomerAsync
    return c;

}).ToList();

Customer[] customers = await Task.WhenAll(customerTasks); //look... all the customers

Of course, there are probably more efficient means of getting several customers in one go, but that would be for a different question.

If instead, you'd like to perform your async tasks in sequence then:

var customerTasks = ids.Select(async i =>
{
    ICustomerRepo repo = new CustomerRepo();
    var id = await repo.getCustomer(i); //consider changing to GetCustomerAsync
    Console.WriteLine(id);

});
foreach(var task in customerTasks) //items in sequence will be materialized one-by-one
{
    await task;
}

Answer 2

Addition:

There seems to be some confusion about when the the LINQ statements actually are executed, especially the Where statement.
I created a small program to show when the source data actually is accessed Results at the end of this answer

end of Addition

You have to be aware about the lazyness of most LINQ functions.

Lazy LINQ functions will only change the Enumerator that IEnumerable.GetEnumerator() will return when you start enumerating. Hence, as long as you call lazy LINQ functions the query isn't executed.

Only when you starte enumerating, the query is executed. Enumerating starts when you call foreach , or non-layzy LINQ functions like ToList() , Any() , FirstOrDefault() , Max() , etc.

In the comments section of every LINQ function is described whether the function is lazy or not. You can also see whether the function is lazy by inspecting the return value. If it returns an IEnumerable<...> (or IQueryable) the LINQ is not enumerated yet.

The nice thing about this lazyness, is that as long as you use only lazy functions, changing the LINQ expression is not time consuming. Only when you use non-lazy functions, you have to be aware of the impact.

For instance, if fetching the first element of a sequence takes a long time to calculate, because of Ordering, Grouping, Database queries etc, make sure you don't start enumerating more then once (= don't use non-lazy functions for the same sequence more than once)

Don't do this at home:

Suppose you have the following query

var query = toDoLists
    .Where(todo => todo.Person == me)
    .GroupBy(todo => todo.Priority)
    .Select(todoGroup => new
    {
        Priority = todoGroup.Key,
        Hours = todoGroup.Select(todo => todo.ExpectedWorkTime).Sum(),
     }
     .OrderByDescending(work => work.Priority)
     .ThenBy(work => work.WorkCount);

This query contains only lazy LINQ functions. After all these statement, the todoLists have not been accessed yet.

But as soon as you get the first element of the resulting sequence all elements have to be accessed (probably more than once) to group them by priority, calculate the total number of involved working hours and to sort them by descending priority.

This is the case for Any(), and again for First():

if (query.Any())                           // do grouping, summing, ordering
{
    var highestOnTodoList = query.First(); // do all work again
    Process(highestOnTodoList);
}
else
{   // nothing to do
    GoFishing();
}

In such cases it is better to use the correct function:

var highestOnToDoList = query.FirstOrDefault(); // do grouping / summing/ ordering
if (highestOnTioDoList != null)
   etc.

back to your question

The Enumerable.Select statement only created an IEnumerable object for you. You forgot to enumerate over it.

Besides you constructed your CustomerRepo several times. Was that intended?

ICustomerRepo repo = new CustomerRepo();
IEnumerable<Task<CustomerRepo>> query = ids.Select(id => repo.getCustomer(i));

foreach (var task in query)
{
     id = await task;
     Console.WriteLine(id);
}

Addition: when are the LINQ statements executed?

I created a small program to test when a LINQ statement is executed, especially when a Where is executed.

A function that returns an IEnumerable:

IEnumerable<int> GetNumbers()
{
    for (int i=0; i<10; ++i)
    {
        yield return i;
    }
}

A program that uses this enumeration using an old fashioned Enumerator

public static void Main()
{
    IEnumerable<int> number = GetNumbers();
    IEnumerable<int> smallNumbers = numbers.Where(number => number < 3);

    IEnumerator<int> smallEnumerator = smallNumbers.GetEnumerator();

    bool smallNumberAvailable = smallEnumerator.MoveNext();
    while (smallNumberAvailable)
    {
        int smallNumber = smallEnumerator.Current;
        Console.WriteLine(smallNumber);
        smallNumberAvailable = smallEnumerator.MoveNext();
    }
}

During debugging I can see that the first time GetNumbers is executed the first time that MoveNext() is called. GetNumbers() is executed until the first yield return statement.

Every time that MoveNext() is called the statements after the yield return are performed until the next yield return is executed.

Changing the code such that the enumerator is accessed using foreach, Any(), FirstOrDefault(), ToDictionary, etc, shows that the calls to these functions are the time that the originating source is actually accessed.

if (smallNumbers.Any())
{
    int x = smallNumbers.First();
    Console.WriteLine(x);
}

Debugging shows that the originating source starts enumerating from the beginning twice. So indeed, it is not wise to do this, especially if you need to do a lot to calculate the first element (GroupBy, OrderBy, Database access, etc)