LINQ sequences - how are they chained in IL?

Question

Whether I use expression query syntax or method syntax the IL looks virtually the same.

Doing the LINQ query as 'progressive syntax':

IEnumerable<Employee> query1 = _employees.Where(e => e.Location.Equals("California"));
IEnumerable<Employee> query2 = query1.OrderByDescending(e => e.Name);
IEnumerable<string> query3 = query2.Select(e => e.Name);

generates:

IL_0001: ldarg.0      // this
IL_0002: ldfld        class [mscorlib]System.Collections.Generic.IList`1<class Fundamentals.LINQ.Employee> Fundamentals.LINQ.Syntax::_employees
IL_0007: ldsfld       class [mscorlib]System.Func`2<class Fundamentals.LINQ.Employee, bool> Fundamentals.LINQ.Syntax/'<>c'::'<>9__3_0'
IL_000c: dup          
IL_000d: brtrue.s     IL_0026
IL_000f: pop          
IL_0010: ldsfld       class Fundamentals.LINQ.Syntax/'<>c' Fundamentals.LINQ.Syntax/'<>c'::'<>9'
IL_0015: ldftn        instance bool Fundamentals.LINQ.Syntax/'<>c'::'<TestMethodSyntaxProgressive>b__3_0'(class Fundamentals.LINQ.Employee)
IL_001b: newobj       instance void class [mscorlib]System.Func`2<class Fundamentals.LINQ.Employee, bool>::.ctor(object, native int)
IL_0020: dup          
IL_0021: stsfld       class [mscorlib]System.Func`2<class Fundamentals.LINQ.Employee, bool> Fundamentals.LINQ.Syntax/'<>c'::'<>9__3_0'
IL_0026: call         class [mscorlib]System.Collections.Generic.IEnumerable`1<!!0/*class Fundamentals.LINQ.Employee*/> [System.Core]System.Linq.Enumerable::Where<class Fundamentals.LINQ.Employee>(class [mscorlib]System.Collections.Generic.IEnumerable`1<!!0/*class Fundamentals.LINQ.Employee*/>, class [mscorlib]System.Func`2<!!0/*class Fundamentals.LINQ.Employee*/, bool>)
IL_002b: stloc.0      // query1

// [100 13 - 100 82]
IL_002c: ldloc.0      // query1
IL_002d: ldsfld       class [mscorlib]System.Func`2<class Fundamentals.LINQ.Employee, string> Fundamentals.LINQ.Syntax/'<>c'::'<>9__3_1'
IL_0032: dup          
IL_0033: brtrue.s     IL_004c
IL_0035: pop          
IL_0036: ldsfld       class Fundamentals.LINQ.Syntax/'<>c' Fundamentals.LINQ.Syntax/'<>c'::'<>9'
IL_003b: ldftn        instance string Fundamentals.LINQ.Syntax/'<>c'::'<TestMethodSyntaxProgressive>b__3_1'(class Fundamentals.LINQ.Employee)
IL_0041: newobj       instance void class [mscorlib]System.Func`2<class Fundamentals.LINQ.Employee, string>::.ctor(object, native int)
IL_0046: dup          
IL_0047: stsfld       class [mscorlib]System.Func`2<class Fundamentals.LINQ.Employee, string> Fundamentals.LINQ.Syntax/'<>c'::'<>9__3_1'
IL_004c: call         class [System.Core]System.Linq.IOrderedEnumerable`1<!!0/*class Fundamentals.LINQ.Employee*/> [System.Core]System.Linq.Enumerable::OrderByDescending<class Fundamentals.LINQ.Employee, string>(class [mscorlib]System.Collections.Generic.IEnumerable`1<!!0/*class Fundamentals.LINQ.Employee*/>, class [mscorlib]System.Func`2<!!0/*class Fundamentals.LINQ.Employee*/, !!1/*string*/>)
IL_0051: stloc.1      // query2

// [101 13 - 101 69]
IL_0052: ldloc.1      // query2
IL_0053: ldsfld       class [mscorlib]System.Func`2<class Fundamentals.LINQ.Employee, string> Fundamentals.LINQ.Syntax/'<>c'::'<>9__3_2'
IL_0058: dup          
IL_0059: brtrue.s     IL_0072
IL_005b: pop          
IL_005c: ldsfld       class Fundamentals.LINQ.Syntax/'<>c' Fundamentals.LINQ.Syntax/'<>c'::'<>9'
IL_0061: ldftn        instance string Fundamentals.LINQ.Syntax/'<>c'::'<TestMethodSyntaxProgressive>b__3_2'(class Fundamentals.LINQ.Employee)
IL_0067: newobj       instance void class [mscorlib]System.Func`2<class Fundamentals.LINQ.Employee, string>::.ctor(object, native int)
IL_006c: dup          
IL_006d: stsfld       class [mscorlib]System.Func`2<class Fundamentals.LINQ.Employee, string> Fundamentals.LINQ.Syntax/'<>c'::'<>9__3_2'
IL_0072: call         class [mscorlib]System.Collections.Generic.IEnumerable`1<!!1/*string*/> [System.Core]System.Linq.Enumerable::Select<class Fundamentals.LINQ.Employee, string>(class [mscorlib]System.Collections.Generic.IEnumerable`1<!!0/*class Fundamentals.LINQ.Employee*/>, class [mscorlib]System.Func`2<!!0/*class Fundamentals.LINQ.Employee*/, !!1/*string*/>)
IL_0077: stloc.2      // query3

According to the way LINQ works is that the output sequence from the Where() operation should become the input sequence to OrderByDescending(), and the 2nd output sequence should flow into the Select(). I just can't understand which IL codes correspond to that.

Is this the magic part?

IL_002b: stloc.0      // query1

// [100 13 - 100 82]
IL_002c: ldloc.0      // query1

If I do the LINQ query as 'fluent':

IEnumerable<string> names = _employees.Where(e => e.Location.Equals("California")).OrderByDescending(e => e.Name).Select(e => e.Name);

I do not get the stloc.0, ldloc.0 instructions. That's the only difference in the IL emitted between 'progressive syntax' & 'fluent syntax'. Is there a second step in the JIT-compiler that generates extra IL instructions that do the sequence inputting?

Answer 1

The only difference between two versions is locals. In the first version, the return value of each call is saved into a local, then the next method is called on that local and so on.

I assume what you are asking is how does it work when there is no locals? Where does the return value of a method stored?

It is stored in the evaluation stack. And the next method is called using the value in the evaluation stack. After the last method is called in the chain the result is set back to the local variable names .

Take a look at the Call instruction to see how it works step by step:

1. Method arguments arg1 through argN are pushed onto the stack.

2. Method arguments arg1 through argN are popped from the stack; the method call is performed with these arguments and control is transferred to the method referred to by the method descriptor. When complete, a return value is generated by the callee method and sent to the caller.

3. The return value is pushed onto the stack.

So before a method is called it's arguments are pushed onto the stack. In your first code it goes like this:

1. Push _employees onto the stack, Push Func<Employee, bool> onto the stack
2. Call Where
3. Store result in local 0
4. Load local 0 (pushing it onto stack), Push Func<Employee, string> onto the stack
5. Call OrderByDescending
6. Store result in local 1
7. Load local 1 , Push Func<Employee, string> onto the stack
8. Call Select
9. Store result in local2

There is no load local stuff in the second version because the return value of a method call is already stored in the stack. When a method is called you have the return value that will become the first argument of the next call, only the delegate which is the second argument is pushed onto the stack and the next method is called.

I should also mention this is not at all specific to LINQ. Method chaining works this way, LINQ is not special.

Note: I have simplified pushing the delegate steps, the extra code there is for caching. The delegate instances are cached in a compiler generated class' fields, before creating a new delegate instance compiler checks whether it's created before for efficiency.