并发处理项目并顺序写入结果？

Question

The following Python code (from Read in parallel and write sequentially? ) processes (reads) the data for ID 0 to N. There readers run in parallel and it writes the result in the order of ID. How to implement it in C# efficiently?

async def reader(q, id_to_data, data_ready):  # Multiple readers are run in parallel
    while True:
        id = await q.get()
        data = await read_async(id) 
        id_to_data[id] = data
        data_ready.set()

async def main():
    q = asyncio.Queue()
    for id in range(1000):
        await q.put(id)

    id_to_data = {}
    data_ready = asyncio.Event()
    readers = [asyncio.create_task(reader(q, id_to_data, data_ready))
               for _ in range(3)]

    for id in range(1000):
       while True:
           # wait for the current ID to appear before writing
           if id in id_to_data:
               data = id_to_data.pop(id)
               await data.write_async(f'{id}.csv')
               break
               # move on to the next ID
           else:
               # wait for new data and try again
               await data_ready.wait()
               data_ready.clear()

    for r in readers:
        r.cancel()

C#:

void main() {
    List<int> ids = get_IDs();
    ProcessAllIDs(ids);
}

async Task ProcessAllIDs(List<int> ids) {
    // A low efficient implementation
    for (var id in ids) {
        var result = await Process1(id); // Can be run concurrently
        await Process2(result); // Need to be run sequentially 
    }
}

Answer 1

A simple way would be to create a List of Tasks and then await in them in sequence:

var list = new List<Task<int>> { Calculate(2), Calculate(7), Calculate(19) };

foreach(var task in list)
{
  Console.WriteLine(await task);
}

static async Task<int> Calculate(int x)
{
  await Task.Delay(200);
  return x*x;
}

This will start 3 Tasks that calculate stuff simultaneously and then print the results in order.

So for your example, it would look like this:

async Task main() {
    List<int> ids = get_IDs();
    await ProcessAllIDs(ids);
}

async Task ProcessAllIDs(List<int> ids) {
    var tasks = ids.Select(id => Process1(id)).ToList();
    for (var t in tasks) {
        await Process2(await t); // Need to be run sequentially 
    }
}