[英]MaxDegreeOfParallelism with Task.Factory.StartNew()
我有一个程序,可以从Internet下载文件并进行处理。 以下是我编写的使用线程下载文件的功能。
Task<File> re = Task.Factory.StartNew(() => { /*Download the File*/ });
re.ContinueWith((x) => { /*Do another function*/ });
我现在希望它仅使用10个线程进行下载。 我已经查看了ParallelOptions.MaxDegreeOfParallelism属性,但是当任务返回结果时,我不明白如何使用它。
您可以使用类似:
Func<File> work = () => {
// Do something
File file = ...
return file
};
var maxNoOfWorkers = 10;
IEnumerable<Task> tasks = Enumerable.Range(0, maxNoOfWorkers)
.Select(s =>
{
var task = Task.Factory.StartNew<File>(work);
return task.ContinueWith(ant => { /* do soemthing else */ });
});
如果您确实想创建一个专用( non-threadpool
线程non-threadpool
)线程,则TPL
决定从线程threadpool
获取多少线程,然后可以使用以下方法:
IEnumerable<Task> tasks = Enumerable.Range(0, maxNoOfWorkers)
.Select(s =>
{
var task = Task.Factory.StartNew<File>(
work,
CancellationToken.None,
TaskCreationOptions.LongRunning,
TaskScheduler.Default);
return task.ContinueWith(ant => { /* do soemthing else */ });
});
您的其他选择是使用PLINQ
或Paraller.For/ForEach
,可以将其与MaxDegreeOfParallelism
一起使用。
PLINQ
示例可以是:
Func<File> work = () => {
// Do something
File file = ...
return file
};
var maxNoOfWorkers = 10;
ParallelEnumerable.Range(0, maxNoOfWorkers)
.WithDegreeOfParallelism(maxNoOfWorkers)
.ForAll(x => {
var file = work();
// Do something with file
});
当然,我不知道您的示例的上下文,因此您可能需要使其适应您的要求。
做到这一点的一种好方法是使用DataFlow API 。 要使用它,您必须安装Microsoft.Tpl.Dataflow Nuget软件包 。
假设您具有以下下载和处理数据的方法:
public async Task<DownloadResult> DownloadFile(string url)
{
//Asynchronously download the file and return the result of the download.
//You don't need a thread to download the file if you use asynchronous API.
}
public ProcessingResult ProcessDownloadResult(DownloadResult download_result)
{
//Synchronously process the download result and produce a ProcessingResult.
}
并假设您具有要下载的URL列表:
List<string> urls = new List<string>();
然后,您可以使用DataFlow API执行以下操作:
TransformBlock<string,DownloadResult> download_block =
new TransformBlock<string, DownloadResult>(
url => DownloadFile(url),
new ExecutionDataflowBlockOptions
{
//Only 10 asynchronous download operations
//can happen at any point in time.
MaxDegreeOfParallelism = 10
});
TransformBlock<DownloadResult, ProcessingResult> process_block =
new TransformBlock<DownloadResult, ProcessingResult>(
dr => ProcessDownloadResult(dr),
new ExecutionDataflowBlockOptions
{
//We limit the number of CPU intensive operation
//to the number of processors in the system.
MaxDegreeOfParallelism = Environment.ProcessorCount
});
download_block.LinkTo(process_block);
foreach(var url in urls)
{
download_block.Post(url);
}
声明:本站的技术帖子网页,遵循CC BY-SA 4.0协议,如果您需要转载,请注明本站网址或者原文地址。任何问题请咨询:yoyou2525@163.com.