How can I extend IEnumerable<T> to load my collection from a web service?

Question

I've been trying to puzzle out how I could implement Enumerable<T> / IEnumerator<T> to satisfy the following:

1. I call a web service to create a query
2. I poll the web service to find out when it is ready to return results
3. Once the web service is ready to return results, I call a method (eg GetNext(n) ) to get n items from it
4. Once GetNext(n) returns less than n results I have retrieved all of my items.

I'm having some difficulty trying to figure out how I could make an IEnumerable handle all of the heavy lifting from me and the MSDN IEnumerable documentation has not helped.

Here is a simplified version of my code:

public class MyEnumerable : IEnumerable
{
    private MyWebService _service;
    private int _queryID;

    public MyEnumerable(MyWebService service, int queryID)
    {
        _service = service;
        _queryID = queryID;
    }

    IEnumerator IEnumerable.GetEnumerator()
    {
        return new MyEnumerator(_service, _query);
    }
}

public class MyEnumerator : IEnumerator
{
    private List<QueryResult> _items;  //want to load my items from WebService into this
    private MyWebService _service;
    private int _queryID;
    private int _index = 0;

    private MyEnumerator(MyWebService service, int queryID)
    {
        _service = service;
        _queryID = queryID;
    }

    public object Current
    {
        //what goes here? 
    }

    public bool MoveNext()
    {
        //what goes here? would I call the web service here?
    }

    public void Reset()
    {
        _index = 0;
    }

}

Hopefully this makes a bit more sense. Let's assume I can call a method on my webservice called GetNext(n).

Answer 1

Rather than explicitly implementing the interface, unless you have a reason not to, you should generally use an iterator block to create such sequences. It will take care of the boilerplate code, just leaving you with the interesting implementation. A problem like this will generally look something along these lines:

//you can make this private if you want
public static IEnumerable<IList<Foo>> GetPages(int pageSize)
{
    IList<Foo> nextGroup = ExecuteQuery();
    while (nextGroup.Count == pageSize)
    {
        yield return nextGroup;
        nextGroup = ExecuteQuery();
    }
    if (nextGroup.Any())
        yield return nextGroup;
}

You can then use SelectMany to flatten out the list of groups into a list of the individual items:

public static IEnumerable<Foo> GetItems(int pageSize)
{
    return GetPages(pageSize).SelectMany(x => x);
}

Answer 2

Use Rx Reactive Extensions, and the Buffer extension method. (Meaning, IObservable rather than IEnumberable). Seems to match your use case perfectly!