[英]Why is there a performance difference between LINQ (c#) vs Seq (f#)
I made very simple C# and F# test programs. 我做了非常简单的C#和F#测试程序。
using System;
using System.Collections.Generic;
using System.Diagnostics;
using System.Linq;
namespace ConsoleApplication2
{
class Program
{
static int Remainder(int num)
{
return num % 2;
}
static int SumOfremainders(IEnumerable<int> list)
{
var sum = 0;
foreach (var num in list)
{
sum += Remainder(num);
}
return sum;
}
static void Main(string[] args)
{
Stopwatch sw = new Stopwatch();
var nums = Enumerable.Range(1, 10000000);
sw.Start();
var a = SumOfremainders(nums);
sw.Stop();
Console.WriteLine("Duration " + (sw.ElapsedMilliseconds));
Console.WriteLine("Sum of remainders: {0}", a);
}
}
}
let remainder x = x % 2
let sumORemainders n =
n
|> Seq.map(fun n-> remainder n)
|> Seq.sum
let seqb = Seq.init 10000000(fun n->n)
let timer =System.Diagnostics.Stopwatch()
timer.Start()
let a =(sumORemainders seqb )
timer.Stop()
printfn "Elapsed Time: "
System.Console.WriteLine timer.ElapsedMilliseconds
printfn "Sum of squares of 1-100: %d" a
[<EntryPoint>]
let main argv =
0 // return an integer exit code
c# 71 ms f# 1797 ms c#71 ms f#1797 ms
I made second version from F# which work similar than c# 我从F#制作了第二个版本,它的工作方式与c#相似
let remainder x = x % 2
let sumORemainders (input:seq<int>) =
let mutable sum = 0
let en = input.GetEnumerator()
while (en.MoveNext()) do
sum <- sum + remainder en.Current
sum
let seqb = Seq.init 10000000(fun n->n)
let timer =System.Diagnostics.Stopwatch()
timer.Start()
let a =(sumORemainders seqb )
timer.Stop()
printfn "Elapsed Time: "
System.Console.WriteLine timer.ElapsedMilliseconds
printfn "Sum of squares of 1-100: %d" a
[<EntryPoint>]
let main argv =
0 // return an integer exit code
but the result not not changed significantly (1650ms) 但结果并没有显着改变(1650ms)
I don't understand the big difference in speed between the two languages. 我不明白两种语言之间速度的巨大差异。
The two programs have very similar IL code, both use IEnumerable, moreover the F# replace the function call with operation. 这两个程序有非常相似的IL代码,都使用IEnumerable,而且F#用操作替换函数调用。
I rewrote c# code based on f# IL code. 我根据f#IL代码重写了c#代码。
static int SumOfremainders(IEnumerable<int> list)
{
var sum = 0;
IEnumerator<int> e = list.GetEnumerator();
while (e.MoveNext())
{
sum += e.Current % 2;
}
return sum;
}
The IL code of two programs are same but the speed still very different. 两个程序的IL代码相同但速度仍然非常不同。 I found the IL difference thanks for Foggy Finder
感谢Foggy Finder,我找到了IL差异
The slow code 慢代码
[CompilationMapping(SourceConstructFlags.Module)]
public static class Program
{
[Serializable]
internal class seqb@18 : FSharpFunc<int, int>
{
internal seqb@18()
{
}
public override int Invoke(int n)
{
return n;
}
}
[CompilationMapping(SourceConstructFlags.Value)]
public static IEnumerable<int> seqb
{
get
{
return $Program.seqb@18;
}
}
The fast code 快速的代码
[CompilationMapping(SourceConstructFlags.Module)]
public static class Program
{
[CompilationMapping(SourceConstructFlags.Value)]
public static int[] seqb
{
get
{
return $Program.seqb@20;
}
}
The primary reason OP sees a performance difference is because Seq.init
in F# is slow. OP看到性能差异的主要原因是因为F#中的
Seq.init
很慢。 The reason for it is that in each iteration Seq.upto
(which Seq.init
uses) allocates a new Lazy<_>
object. 原因是在每次迭代中
Seq.upto
( Seq.init
使用)分配一个新的Lazy<_>
对象。 You can see this in Seq
source . 您可以在
Seq
源中看到这一点。 If you have a low overhead function like fun n -> n % 2
the cost of a new Lazy<_>
object as well as evaluating it (mutex lock and unlock) takes a significant amount of time. 如果你有一个低开销函数,如
fun n -> n % 2
,新的Lazy<_>
对象的成本以及评估它(互斥锁和解锁)需要花费大量的时间。
The second reason OP sees a performance difference is that Seq
in F# is slow in general. OP看到性能差异的第二个原因是F#中的
Seq
一般都很慢。 This is being remedied by manofstick in this PR 这个PR正在通过manofstick来解决
With the PR inplace F# Seq
will perform very well compared to the alternatives that exists (some details here ) 随着PR inplace F#
Seq
与现有的替代品相比表现非常好( 这里有一些细节)
With all this said I prepared some comparisons of different ways to do the computation posted by the user (apart from the obvious total / 2
). 有了这一切,我准备了一些比较不同的方法来进行用户发布的计算(除了显而易见的
total / 2
)。
open CsPerfs
open Nessos.Streams
open System.Diagnostics
open System.Linq
open System.Numerics
// now () returns current time in milliseconds since start
let now : unit -> int64 =
let sw = Stopwatch ()
sw.Start ()
fun () -> sw.ElapsedMilliseconds
// time estimates the time 'action' repeated a number of times
let time repeat action =
let inline cc i = System.GC.CollectionCount i
let v = action ()
System.GC.Collect (2, System.GCCollectionMode.Forced, true)
let bcc0, bcc1, bcc2 = cc 0, cc 1, cc 2
let b = now ()
for i in 1..repeat do
action () |> ignore
let e = now ()
let ecc0, ecc1, ecc2 = cc 0, cc 1, cc 2
v, (e - b), ecc0 - bcc0, ecc1 - bcc1, ecc2 - bcc2
[<EntryPoint>]
let main argv =
let count = 10000000
let outers =
[|
1000
|]
for outer in outers do
let inner = count / outer
let fsImperativeTest () =
let mutable sum = 0
for n = 0 to inner-1 do
sum <- sum + n % 2
sum
let fsLinqTest () =
Enumerable.Range(0, inner).Select(fun n -> n % 2).Sum()
let fsNessosTest () =
Stream.initInfinite id
|> Stream.take inner
|> Stream.map (fun n -> n % 2)
|> Stream.sum
let fsOpTest () =
let remainder x = x % 2
let sumORemainders (input:seq<int>) =
let mutable sum = 0
use en = input.GetEnumerator()
while (en.MoveNext()) do
sum <- sum + remainder en.Current
sum
let seqb = Seq.init inner id
sumORemainders seqb
let fsSseTest () =
let inc = Vector<int>.One
let one = Vector<int>.One
let mutable sum = Vector<int>.Zero
let mutable n = Vector<int> [|0..3|]
for n4 = 0 to inner/4-1 do
n <- n + inc
sum <- sum + (n &&& one)
sum.[0] + sum.[1] + sum.[2] + sum.[3]
let fsSeqTest () =
Seq.init inner id
|> Seq.map (fun n -> n % 2)
|> Seq.sum
let fsSeqVariantTest () =
seq { for n = 0 to inner-1 do yield n }
|> Seq.map (fun n -> n % 2)
|> Seq.sum
let csImperativeTest =
let f = Perfs.CsImperativeTest inner
fun () -> f.Invoke ()
let csLinqTest =
let f = Perfs.CsLinqTest inner
fun () -> f.Invoke ()
let csOpTest =
let f = Perfs.CsOpTest inner
fun () -> f.Invoke ()
let tests =
[|
"fsImperativeTest" , fsImperativeTest
"fsLinqTest" , fsLinqTest
"fsNessosTest" , fsNessosTest
"fsOpTest" , fsOpTest
"fsSeqTest" , fsSeqTest
"fsSeqVariantTest" , fsSeqVariantTest
"fsSseTest" , fsSseTest
"csImperativeTest" , csImperativeTest
"csLinqTest" , csLinqTest
"csOpTest" , csOpTest
|]
printfn "Test run - total count: %d, outer: %d, inner: %d" count outer inner
for name, test in tests do
printfn "Running %s..." name
let v, ms, cc0, cc1, cc2 = time outer test
printfn " it took %d ms - collection count is %d,%d,%d - result is %A" ms cc0 cc1 cc2 v
0
The matching C# code: 匹配的C#代码:
namespace CsPerfs
{
using System;
using System.Collections.Generic;
using System.Linq;
public static class Perfs
{
static int Remainder(int num)
{
return num % 2;
}
static int SumOfremainders(IEnumerable<int> list)
{
var sum = 0;
foreach (var num in list)
{
sum += Remainder(num);
}
return sum;
}
public static Func<int> CsOpTest (int count)
{
return () => SumOfremainders (Enumerable.Range(1, count));
}
public static Func<int> CsImperativeTest (int count)
{
return () =>
{
var sum = 0;
for (var n = 0; n < count; ++n)
{
sum += n % 2;
}
return sum;
};
}
public static Func<int> CsLinqTest (int count)
{
return () => Enumerable.Range (0, count).Select (n => n % 2).Sum ();
}
}
}
The performance numbers seen on my machine (Intel Core I5) running on .NET 4.6.1 64bit: 在我的机器(Intel Core I5)上运行的.NET 4.6.1 64位上的性能数字:
Test run - total count: 10000000, outer: 1000, inner: 10000
Running fsImperativeTest...
it took 20 ms - collection count is 0,0,0 - result is 5000
Running fsLinqTest...
it took 124 ms - collection count is 0,0,0 - result is 5000
Running fsNessosTest...
it took 56 ms - collection count is 0,0,0 - result is 5000
Running fsOpTest...
it took 1320 ms - collection count is 661,0,0 - result is 5000
Running fsSeqTest...
it took 1477 ms - collection count is 661,0,0 - result is 5000
Running fsSeqVariantTest...
it took 512 ms - collection count is 0,0,0 - result is 5000
Running fsSseTest...
it took 2 ms - collection count is 0,0,0 - result is 5000
Running csImperativeTest...
it took 19 ms - collection count is 0,0,0 - result is 5000
Running csLinqTest...
it took 122 ms - collection count is 0,0,0 - result is 5000
Running csOpTest...
it took 58 ms - collection count is 0,0,0 - result is 5000
Press any key to continue . . .
Seq
does the worst and also consumes memory. Seq
做得最差,也消耗内存。 If both F# and C# code uses LINQ there's no real difference as expected. 如果F#和C#代码都使用LINQ,则没有预期的真正差异。 Nessos is a high-performance data pipeline for F# (and C#) that does significantly better.
Nessos是F#(和C#)的高性能数据管道,它做得更好。
"Hard-coding" a for loop does better still and the fastest solution is to use SSE through System.Numerics.Vectors
. “硬编码”for循环确实更好,最快的解决方案是通过
System.Numerics.Vectors
使用SSE。 Unfortunately System.Numerics.Vectors
doesn't support %
which makes the comparison a bit unfair. 不幸的是,
System.Numerics.Vectors
不支持%
,这使得比较有点不公平。
So the difference is not so much a language issue but a library issue. 所以区别不仅仅是语言问题,而是图书馆问题。
声明:本站的技术帖子网页,遵循CC BY-SA 4.0协议,如果您需要转载,请注明本站网址或者原文地址。任何问题请咨询:yoyou2525@163.com.