Fix for C# Async Method Executing Sequentially

Question

The problem I'm running into is a call to an async method I'm performing is happening sequentially. I'm adding the call's tasks to a ConcurrentBag and awaiting the tasks in the bag. I don't care about the results of these calls, I just need confirmation that they completed. However, these calls are happening fully sequentially which is very confusing. The method in question performs a few PostgreSQL queries via Npgsql with parameterized queries. The caller gets a tree of our own data and pulls out all the nodes in the tree and iterates over the nodes and performs this task on them. I am also using a custom AsyncHelper class which will iterate over the tasks in an IEnumerable implementer and wait the tasks inside it. Both my Tree implementation and AsyncHelper have been tested in another piece of code that does the same basic principles of this code, which executes the tasks asynchronously as expected.

I've added logging on the function calls to confirm these are happening sequentially. I event also taking the method out of the bag and just running the method, which still does the same thing, it happens sequentially and won't continue my loop until it's done. All my methods are labeled async and I'm not awaiting them until after the loop.

//method executing sequentially
public static async Task<List<ContactStatistic>> getContactStats(Guid tenantId, DateTime start, DateTime end, Breakdown breakdown) {
    if (!await Postgres.warmConnection(5)) { return null; }
    var hierarchy = await getTreeForTenant<TenantContactStatsNode>(tenantId);

    //perform calculations to determine stats for each element
    var calculationTasks = new ConcurrentBag<Task>();
    var allData = await hierarchy.getAllData();
    var timestampGotAllData = DateTime.Now;

    foreach (var d in allData) {
        calculationTasks.Add(d.getContactStats(start, end, breakdown));
    }

    Console.WriteLine("about to await all the tasks");
    //await the tasks to complete for calculations
    await AsyncHelper.waitAll(calculationTasks);
}


//method it's calling
public async Task getContactStats(DateTime start, DateTime end, Breakdown breakdown) {
    //perform two async postgres calls
    //await postgres calls
    //validate PG response
    //perform manipluation on this object with data from the queries
}

I would expect the first call to call the second function, add the task to the bag, and wait on them after it's done. What's actually occurring is that the method is running, finishing, then adding to the bag.

* EDIT *

Below is the full code for that second call as requested. It takes in some data from the database based on time, fills the gaps between the times pulled back so we have a fully sequential return list including all times not having data in the database, and puts that in a object level variable

public async Task getContactStats(DateTime start, DateTime end, Breakdown breakdown) {
    if (breakdown == Breakdown.Month) {
        //max out month start day to include all for the initial month in the initial count
        start = new DateTime(start.Year, start.Month, DateTime.DaysInMonth(start.Year, start.Month));
    } else {
        //day breakdown previous stats should start the day before given start day
        start = start.AddDays(-1);
    }

    var tran = new PgTran();
    var breakdownQuery = breakdown == Breakdown.Day ? Queries.GET_CONTACT_DAY_BREAKDOWN : Queries.GET_CONTACT_MONTH_BREAKDOWN;
    tran.setQueries(Queries.GET_CONTACT_COUNT_BEFORE_DATE, breakdownQuery);
    tran.setParams(new NpgsqlParameter("@tid", tenantId), new NpgsqlParameter("@start", start), new NpgsqlParameter("@end", end));
    var tranResults = await Postgres.getAll<ContactDayStatistic>(tran);
    //ensure transaction returns two query results
    if (tranResults == null || tranResults.Count != 2) { return; }


    //ensure valid past count was retrieved
    var prevCountResult = tranResults[0];
    if (prevCountResult == null || prevCountResult.Count != 1) { return; }
    var prevStat = new ContactDayStatistic(start.Day, start.Month, start.Year, prevCountResult[0].count);
    //ensure valid contact stat breakdown was retrieved
    var statBreakdown = tranResults[1];
    if (statBreakdown == null) { return;}

    var datesInBreakdown = new List<DateTime?>();
    //get all dates in the returned stats
    foreach (var e in statBreakdown) {
        var eventDate = new DateTime(e.year, e.month, e.day);
        if (datesInBreakdown.Find(item => item == eventDate) == null)
            datesInBreakdown.Add(eventDate);
    }
    //sort so they are sequential
    datesInBreakdown.Sort();

    //initialize timeline starting with initial breakdown
    var fullTimeline = new List<ContactStatistic>();
    //convert initial stat to the right type for final display
    fullTimeline.Add(breakdown == Breakdown.Month ? new ContactStatistic(prevStat) : prevStat);
    foreach (var d in datesInBreakdown) {
        //null date is useless, won't occur, nullable date just for default value of null
        if (d == null) { continue; }
        var newDate = d.Value;
        //fill gaps between last date given and this date
        ContactStatistic.fillGaps(breakdown, newDate, prevStat.getDate(), prevStat.count, ref fullTimeline, false);
        //get stat for this day
        var stat = statBreakdown.Find(item => d == new DateTime(item.year, item.month, item.day));
        if (stat == null) { continue; }
        //add last total for a rolling total of count
        stat.count += prevStat.count;
        fullTimeline.Add(breakdown == Breakdown.Month ? new ContactStatistic(stat) : stat);
        prevStat = stat;
    }
    //fill gaps between last date and end
    ContactStatistic.fillGaps(breakdown, end, prevStat.getDate(), prevStat.count, ref fullTimeline, true);
    //cast list to appropriate return type
    contactStats.Clear();
    contactStats = fullTimeline;
}

* EDIT 2 * Here is the code the AsyncHelper is using to await these tasks. This function works perfectly for other code using this same framework and it's basically just to clean up code that has to wait on enumerated tasks.

public static async Task waitAll(IEnumerable<Task> coll) {
    foreach (var taskToWait in coll) {
        await taskToWait;
    }  
}

* EDIT 3 * As per recommendation, I changed waitAll() to use Task.WhenAll() instead of a foreach loop, the issue is still occurring however.

public static async Task waitAll(IEnumerable<Task> coll) {
    await Task.WhenAll(coll);
}

* EDIT 4 * To ensure it's not the Postgres calls making this happen, I changed the second method to only do a print line then sleep for 200 milliseconds to keep the execution path clear. I still notice that is happening fully sequentially (even causing my POST to this function time out because the actual real call takes almost 20ms). Below is the code for that change to demonstrate

public async Task getContactStats(DateTime start, DateTime end, Breakdown breakdown) {
    Console.WriteLine("CALLED!");
    Thread.Sleep(200);
}

* EDIT 5 * Per recommendation, I tried a parallel foreach to try to populate the ConcurrentBag of tasks rather than a normal foreach. I run into an issue here where the parallel foreach finishes once the first add is done and does not add all of the tasks at once.

var calculationTasks = new ConcurrentBag<Task>();
var allData = await hierarchy.getAllData();
var timestampGotAllData = DateTime.Now;
Parallel.ForEach(allData, item => {
    Console.WriteLine("trying parallel foreach");
    calculationTasks.Add(item.getContactStats(start, end, breakdown));
});

Console.WriteLine("about to await all the tasks");
//await the tasks to complete for calculations
await AsyncHelper.waitAll(calculationTasks);

* EDIT 6 * For visual, I ran the code and did some output to show the weirdness going on. The code executing is as follows:

foreach (var d in allData) {
    Console.WriteLine("Adding call to bag");
    calculationTasks.Add(d.getContactStats(start, end, breakdown));
    Console.WriteLine("Done adding call to bag");
}

The output is: https://i.imgur.com/3y5S4eS.png

Since it's printing "CALLED" every time, then "Done!" before "Done adding call to bag", these executions are happening sequentially, not async as expected.

Answer 1

My gut instinct on this is that it will be something to do with the transaction that you are opening within your method. It's a little hard to tell exactly what is going on within your code as there seem to be a few custom classes here - but is there potentially some locking going on as you open your transaction? As this happens prior to your first await, it will have to run 'sequentially' prior to the code being awaited.

Your custom 'waitall' method doesn't seem to be the problem, but you should consider removing this and using the built in Task.WhenAll to await these asynchronously.

Answer 2

Try this:

foreach (var d in allData) 
{
    calculationTasks.Add(Task.Run(() => d.getContactStats(start, end, breakdown)));
}

//Other code here
//...

Task.WaitAll(calculationTasks.ToArray());

We are essentially creating a task that will "run" your method. We then await for those tasks to complete.

Admittedly, I am not entirely sure why your version blocks, but this seems to do the trick.

UPDATE:

I tested by outputting the thread id and the OP's version executes the tasks on the same thread. Perhaps the thread is being locked by the bag, which forces the new tasks to wait? My proposed solution results on different thread ids which I think explains why it doesn't block.