How does IEnumerable<T> work in background

Question

I am wandering about the more in-depth functionality of the IEnumerable<T> interface.

Basically, it works as an intermediary step in execution. For example, if you write:

IEnumerable<int> temp = new int[]{1,2,3}.Select(x => 2*x);

The result of the Select function will not be calculated (enumerated) until something is done with temp to allow it (such as List<int> list = temp.ToList() ).

However, what puzzles me is, since IEnumerable<T> is an interface, it cannot, by definition, be instantiated. So, what is the collection the actual items (in the example 2*x items) reside in?

Moreover, if we were to write IEnumerable<int> temp = Enumerable.Repeat(1, 10); , what would be the underlying collection where the 1s are stored (array, list, something else)?

I cannot seem to find a thorough (more in-depth) explanation as to the actual implementation of this interface and its functionality (for example, if there is an underlying collection, how does the yield keyword work).

Basically, what I am asking for is a more elaborate explanation on the functionality of IEnumerable<T> .

Answer 1

Implementation shouldn't matter. All these (LINQ) methods return IEnumerable<T> , interface members are the only members you can access, and that should be enough to use them.

However, if you really have to know, you can find actual implementations on http://sourceof.net .

• Enumerable.cs

But, for some of the methods you won't be able to find explicit class declaration, because some of them use yield return , which means proper class (with state machine) is generated by compiler during compilation. eg Enumerable.Repeat is implemented that way:

public static IEnumerable<int> Range(int start, int count) {
    long max = ((long)start) + count - 1;
    if (count < 0 || max > Int32.MaxValue)
        throw Error.ArgumentOutOfRange("count");
    return RangeIterator(start, count); 
}

static IEnumerable<int> RangeIterator(int start, int count) {
    for (int i = 0; i < count; i++)
        yield return start + i;
}

You can read more about that on MSDN: Iterators (C# and Visual Basic)

Answer 2

Not all objects that implement IEnumerable defer execution in some way. The API of the interface makes it possible to defer execution, but it doesn't require it. There are likewise implementations that don't defer execution in any way.

So, what is the collection the actual items (in the example 2*x items) reside in?

There is none. Whenever the next value is requested it computes that one value on demand , gives it to the caller, and then forgets the value. It doesn't store it anywhere else.

Moreover, if we were to write IEnumerable<int> temp = Enumerable.Repeat(1, 10); , what would be the underlying collection where the 1s are stored (array, list, something else)?

There wouldn't be one. It would compute each new value immediately when you ask for the next value and it won't remember it afterward. It only stores enough information to be able to compute the next value, which means it only needs to store the element and the number of values left to yield.

While the actual .NET implementations will use much more concise means of creating such a type, creating an enumerable that defers execution is not particularly hard. Doing so even the long way is more tedious than difficult. You simply compute the next value in the MoveNext method of the iterator. In the example you asked of, Repeat , this is easy as you only need to compute if there is another value, not what it is:

public class Repeater<T> : IEnumerator<T>
{
    private int count;
    private T element;

    public Repeater(T element, int count)
    {
        this.element = element;
        this.count = count;
    }
    public T Current { get { return element; } }

    object IEnumerator.Current
    {
        get { return Current; }
    }

    public void Dispose() { }

    public bool MoveNext()
    {
        if (count > 0)
        {
            count--;
            return true;
        }
        else
            return false;
    }

    public void Reset()
    {
        throw new NotSupportedException();
    }
}

(I've omitted an IEnumerable type that just returns a new instance of this type, or a static Repeat method that creates a new instance of that enumerable. There isn't anything particularly interesting to see there.)

A slightly more interesting example would be something like Count :

public class Counter : IEnumerator<int>
{
    private int remaining;

    public Counter(int start, int count)
    {
        Current = start;
        this.remaining = count;
    }
    public int Current { get; private set; }

    object IEnumerator.Current
    {
        get { return Current; }
    }

    public void Dispose() { }

    public bool MoveNext()
    {
        if (remaining > 0)
        {
            remaining--;
            Current++;
            return true;
        }
        else
            return false;
    }

    public void Reset()
    {
        throw new NotSupportedException();
    }
}

Here we're not only computing if we have another value, but what that next value is, each time a new value is requested of us.

Answer 3

So, what is the collection the actual items (in the example 2*x items) reside in?

It is not residing anywhere. There is code that will produce the individual items "on demand" when you iterate, but the 2*x numbers are not computed upfront. They are also not stored anywhere, unless you call ToList or ToArray .

Moreover, if we were to write IEnumerable temp = Enumerable.Repeat(1, 10);, what would be the underlying collection where the 1s are stored (array, list, something else)?

The same picture is here: the returned implementation of IEnumerable is not public, and it returns its items on demand, without storing them anywhere.

C# compiler provides a convenient way to implement IEnumerable without defining a class for it. All you need is to declare your method return type as IEnumerable<T> , and use yield return to supply values on as-needed basis.