异步 function 调用未执行

Question

我正在尝试使用异步方法将大量文件从一个 S3 复制到另一个 S3。 为了达到同样的目的，将大量文件分成多个批次，每个批次都交给一个异步方法列表。 问题是，每个异步方法在批处理中不处理超过 1 个文件，而每个批处理包含超过 1k 个文件，不确定为什么 async 不返回处理剩余文件。

这是代码：

public void CreateAndExecuteSpawn(string srcBucket, List<List<string>> pdfFileList, IAmazonS3 s3client)
{
    int i = 0;
    List<Action> actions = new List<Action>();
    LambdaLogger.Log("PDF Set count: " + pdfFileList.Count.ToString());
    foreach (var list in pdfFileList)
        actions.Add(() => RenameFilesAsync(srcBucket, list, s3client));

    foreach (var method in actions)
    {
        method.Invoke();
        LambdaLogger.Log("Mehtod invoked: "+ i++.ToString());
    }
}
            
public async void RenameFilesAsync(string srcBucket, List<string> pdfFiles, IAmazonS3 s3client)
{
    LambdaLogger.Log("In RenameFileAsync method");
    CopyObjectRequest copyRequest = new CopyObjectRequest
    {
        SourceBucket = srcBucket,
        DestinationBucket = srcBucket
    };
    try
    {
        foreach (var file in pdfFiles)
        {
            if (!file.Contains("index.xml"))
            {
                string[] newFilename = file.Split('{');
                string[] destKey = file.Split('/');
                copyRequest.SourceKey = file;
                copyRequest.DestinationKey = destKey[0] + "/" + destKey[1] + "/Renamed/" + newFilename[1];
                LambdaLogger.Log("About to rename File: " + file);
                //Here after copying one file, function doesn't return to foreach loop
                CopyObjectResponse response = await s3client.CopyObjectAsync(copyRequest);
                //await s3client.CopyObjectAsync(copyRequest);
                LambdaLogger.Log("Rename done: ");
            }
        }
    }
    catch(Exception ex)
    {
        LambdaLogger.Log(ex.Message);
        LambdaLogger.Log(copyRequest.DestinationKey);
    }
}
        
public void FunctionHandler(S3Event evnt, ILambdaContext context)
{
    //Some code here
    CreateAndExecuteSpawn(bucket, pdfFileSet, s3client);
}

Answer 1

首先，您需要修复批次，以便它一次处理一个批次。 避免async void ； 改用async Task ：

public async Task CreateAndExecuteSpawnAsync(string srcBucket, List<List<string>> pdfFileList, IAmazonS3 s3client)
{
    int i = 0;
    List<Func<Task>> actions = new();
    LambdaLogger.Log("PDF Set count: " + pdfFileList.Count.ToString());
    foreach (var list in pdfFileList)
        actions.Add(() => RenameFilesAsync(srcBucket, list, s3client));

    foreach (var method in actions)
    {
        await method();
        LambdaLogger.Log("Mehtod invoked: "+ i++.ToString());
    }
}
            
public async Task RenameFilesAsync(string srcBucket, List<string> pdfFiles, IAmazonS3 s3client)

然后，您可以在每个批次中添加异步并发。 当前代码只是一个foreach循环，所以当然它一次只处理一个。 您可以通过Select将要运行的任务更改为异步并发，然后在最后执行Task.WhenAll ：

LambdaLogger.Log("In RenameFileAsync method");
CopyObjectRequest copyRequest = new CopyObjectRequest
{
    SourceBucket = srcBucket,
    DestinationBucket = srcBucket
};
try
{
  var tasks = pdfFiles
      .Where(file => !file.Contains("index.xml"))
      .Select(async file =>
      {
         string[] newFilename = file.Split('{');
         string[] destKey = file.Split('/');
         copyRequest.SourceKey = file;
         copyRequest.DestinationKey = destKey[0] + "/" + destKey[1] + "/Renamed/" + newFilename[1];
         LambdaLogger.Log("About to rename File: " + file);
         CopyObjectResponse response = await s3client.CopyObjectAsync(copyRequest);
         LambdaLogger.Log("Rename done: ");
      })
      .ToList();
  await Task.WhenAll(tasks);
}