[英]c# Parallel vs Sequential
I got a large list to loop through (1.500.000 items), with each item I have to do a very small check. 我有一个很大的列表循环(1.500.000项),每个项目我都要做一个非常小的检查。 Totally during 30 seconds.
完全在30秒内。
The CPU utilization when using Sequential is around 10%, so there's a lot of resources not used. 使用Sequential时的CPU利用率约为10%,因此没有使用大量资源。
The first thought was to use Parallel, but due to the limited time duration for each item, Parallel lasts longer than a sequential Foreach, this is due to " Why was the parallel version slower than the sequential version in this example? ", which explains that the creation of each task will cost time. 第一个想法是使用Parallel,但是由于每个项目的持续时间有限,Parallel比持续的Foreach持续时间更长,这是因为“ 为什么并行版本比这个例子中的顺序版本慢? ”,这解释了每项任务的创建都会花费时间。
So I had another thought and that is to divide the list in 4 (or more) equal peaces and create a thread to loop through the items to get it faster. 所以我有另一个想法,那就是将列表分成4个(或更多)相等的和平并创建一个线程来遍历项目以使其更快。
Before creating my own class, is this a good approach? 在创建自己的课程之前,这是一个好方法吗? Or any other thoughts of how to speed things up?
或者关于如何加快速度的任何其他想法? Or do you know a better way of handling this.
或者你知道更好的处理方法吗?
The code I created for another parallel approach: (used in my own static class) 我为另一个并行方法创建的代码:(在我自己的静态类中使用)
public static void ForEach<T>(IEnumerable<T> list, Action<T> body, int listDevide)
{
// Number of items
int items = list.Count();
// Divided (in int, so floored)
int listPart = items / listDevide;
// Get numbers extra for last run
int rest = items % listDevide;
// List to save the actions
var actions = new List<Action>();
for(var x = 0; x < listDevide; x++)
{
// Create the actions
actions.Add(delegate {
foreach(var item in list.Skip(x * listPart).Take(listPart))
{
body.Invoke(item);
}
});
}
// Run the actions parallel
Parallel.Invoke(actions.ToArray());
}
Remark: "rest" variable to do the last items is currently not used in this example. 备注:此示例中当前未使用“rest”变量来执行最后一项。
Solution below, more info: http://msdn.microsoft.com/en-us/library/dd997411.aspx 解决方案如下,更多信息: http : //msdn.microsoft.com/en-us/library/dd997411.aspx
Yes, partitioning the input array is a good approach. 是的,对输入数组进行分区是一种很好的方法。
In fact, Microsoft provide a Partitioner
class to help with exactly this approach. 实际上,Microsoft提供了一个
Partitioner
类来帮助完成这种方法。
Here's an example showing how to do it: 这是一个展示如何操作的示例:
using System;
using System.Collections.Concurrent;
using System.Collections.Generic;
using System.Diagnostics;
using System.Threading.Tasks;
namespace Demo
{
class Program
{
private void run()
{
double sum = 0;
Func<double, double> func = x => Math.Sqrt(Math.Sin(x));
object locker = new object();
double[] data = testData();
// For each double in data[] we are going to calculate Math.Sqrt(Math.Sin(x)) and
// add all the results together.
//
// To do this, we use class Partitioner to split the input array into just a few partitions,
// (the Partitioner will use knowledge about the number of processor cores to optimize this)
// and then add up all the values using a separate thread for each partition.
//
// We use threadLocalState to compute the total for each partition, and then we have to
// add all these together to get the final sum. We must lock the additon because it isn't
// threadsafe, and several threads could be doing it at the same time.
Parallel.ForEach
(
Partitioner.Create(0, data.Length),
() => 0.0,
(subRange, loopState, threadLocalState) =>
{
for (int i = subRange.Item1; i < subRange.Item2; i++)
{
threadLocalState += func(data[i]);
}
return threadLocalState;
},
finalThreadLocalState =>
{
lock (locker)
{
sum += finalThreadLocalState;
}
}
);
Console.WriteLine("Sum = " + sum);
}
private static double[] testData()
{
double[] array = new double[1000003]; // Test with an odd number of values.
Random rng = new Random(12345);
for (int i = 0; i < array.Length; ++i)
array[i] = rng.Next() & 3; // Don't want large values for this simple test.
return array;
}
static void Main()
{
new Program().run();
}
}
}
声明:本站的技术帖子网页,遵循CC BY-SA 4.0协议,如果您需要转载,请注明本站网址或者原文地址。任何问题请咨询:yoyou2525@163.com.