I have a Parallel.ForEach() async loop with which I download some webpages. My bandwidth is limited so I can download only x pages per time but Parallel.ForEach executes whole list of desired webpages.
Is there a way to limit thread number or any other limiter while running Parallel.ForEach?
Demo code:
Parallel.ForEach(listOfWebpages, webpage => {
Download(webpage);
});
The real task has nothing to do with webpages, so creative web crawling solutions won't help.
You can specify a MaxDegreeOfParallelism
in a ParallelOptions
parameter:
Parallel.ForEach(
listOfWebpages,
new ParallelOptions { MaxDegreeOfParallelism = 4 },
webpage => { Download(webpage); }
);
MSDN: Parallel.ForEach
You can use ParallelOptions and set MaxDegreeOfParallelism to limit the number of concurrent threads:
Parallel.ForEach(
listOfwebpages,
new ParallelOptions{MaxDegreeOfParallelism=2},
webpage => {Download(webpage);});
使用采用ParallelOptions
实例的Parallel.Foreach
另一个重载,并设置MaxDegreeOfParallelism
以限制并行执行的实例数量。
And for the VB.net users (syntax is weird and difficult to find)...
Parallel.ForEach(listOfWebpages, New ParallelOptions() With {.MaxDegreeOfParallelism = 8}, Sub(webpage)
......end sub)
The technical post webpages of this site follow the CC BY-SA 4.0 protocol. If you need to reprint, please indicate the site URL or the original address.Any question please contact:yoyou2525@163.com.