[英]Parallel.ForEachAsync keep sort order
I am trying to execute file upload using Parallel.ForEachAsync
, it works but loses the sort order.我正在尝试使用
Parallel.ForEachAsync
执行文件上传,它有效但丢失了排序顺序。 Is there any method to synchronize sort order or source and destination lists?有什么方法可以同步排序顺序或源和目标列表吗?
await Parallel.ForEachAsync(model.DestinationFiles,
new ParallelOptions { MaxDegreeOfParallelism = 20 }, async (file, CancellationToken) =>
{
var storeAsync = await _fileServerService.Init(displayUrl).StoreAsync(file.FileInfo, false, file.OutputFileName);
convertResultDto.Files.Add(new ConverterConvertResultFile(storeAsync));
});
Previously I used Linq parallel operator (PLINQ), which has the AsOrdered
operator to deal with sorting.之前我用过Linq并行运算符(PLINQ),它有
AsOrdered
运算符来处理排序。 Anyway, I think the Parallel.ForEachAsync
is better for using in async methods with I/O scenario?无论如何,我认为
Parallel.ForEachAsync
更适合在具有 I/O 场景的异步方法中使用?
var storeFiles = model.DestinationFiles.AsParallel().AsOrdered().WithDegreeOfParallelism(50)
.Select(file => StoreAsync(file.FileInfo, false, file.OutputFileName).GetAwaiter().GetResult())
.Select(storeFile => new StoreFile
{
FileId = storeFile.FileId,
Url = storeFile.Url,
OutputFileName = storeFile.OutputFileName,
Size = storeFile.Size
});
In this case, you're wanting to get a set of results and store them in a resulting collection.在这种情况下,您想要获得一组结果并将它们存储在结果集合中。
Parallel
is designed for more operations without results. Parallel
是为没有结果的更多操作而设计的。 For operations with results, you can use PLINQ for CPU-bound operations or asynchronous concurrency for I/O-bound operations.对于有结果的操作,您可以将 PLINQ 用于 CPU 绑定操作或异步并发用于 I/O 绑定操作。 Unfortunately, there isn't a PLINQ equivalent for
Parallel.ForEachAsync
, which would be the closest equivalent to your current code.不幸的是,没有
Parallel.ForEachAsync
的 PLINQ 等价物,这将是最接近您当前代码的等价物。
Asynchronous concurrency uses Task.WhenAll
to get the results of multiple asynchronous operations.异步并发使用
Task.WhenAll
来获取多个异步操作的结果。 It can also use SemaphoreSlim
for throttling.它还可以使用
SemaphoreSlim
进行节流。 Something like this:是这样的:
var mutex = new SemaphoreSlim(20);
var results = await Task.WhenAll(model.DestinationFiles.Select(async file =>
{
await mutex.WaitAsync();
try
{
var storeAsync = await _fileServerService.Init(displayUrl).StoreAsync(file.FileInfo, false, file.OutputFileName);
return new ConverterConvertResultFile(storeAsync);
}
finally { mutex.Release(); }
});
convertResultDto.Files.AddRange(results);
However, if you have a mixture of CPU-bound and I/O-bound operations, then you'll probably want to continue to use ForEachAsync
.但是,如果您混合使用受 CPU 限制和 I/O 限制的操作,那么您可能希望继续使用
ForEachAsync
。 In that case, you can create the entries in your destination collection first, then perform each operation with an index so it knows where to store them:在这种情况下,您可以先在目标集合中创建条目,然后使用索引执行每个操作,以便它知道将它们存储在哪里:
// This code assumes convertResultDto.Files is empty at this point.
var count = model.DestinationFiles.Count;
convertResultDto.Files.AddRange(Enumerable.Repeat<ConverterConvertResultFile>(null!, count));
await Parallel.ForEachAsync(
model.DestinationFiles.Select((file, i) => (file, i)),
new ParallelOptions { MaxDegreeOfParallelism = 20 },
async item =>
{
var (file, i) = item;
var storeAsync = await _fileServerService.Init(displayUrl).StoreAsync(file.FileInfo, false, file.OutputFileName);
convertResultDto.Files[i] = new ConverterConvertResultFile(storeAsync);
});
声明:本站的技术帖子网页,遵循CC BY-SA 4.0协议,如果您需要转载,请注明本站网址或者原文地址。任何问题请咨询:yoyou2525@163.com.