“Possible multiple enumeration of IEnumerable” vs “Parameter can be declared with base type”

Question

In Resharper 5, the following code led to the warning "Parameter can be declared with base type" for list :

public void DoSomething(List<string> list)
{
    if (list.Any())
    {
        // ...
    }
    foreach (var item in list)
    {
        // ...
    }
}

In Resharper 6, this is not the case. However, if I change the method to the following, I still get that warning:

public void DoSomething(List<string> list)
{
    foreach (var item in list)
    {
        // ...
    }
}

The reason is, that in this version, list is only enumerated once, so changing it to IEnumerable<string> will not automatically introduce another warning. Now, if I change the first version manually to use an IEnumerable<string> instead of a List<string> , I will get that warning ("Possible multiple enumeration of IEnumerable") on both occurrences of list in the body of the method:

public void DoSomething(IEnumerable<string> list)
{
    if (list.Any()) // <- here
    {
        // ...
    }
    foreach (var item in list) // <- and here
    {
        // ...
    }
}

I understand, why, but I wonder, how to solve this warning, assuming, that the method really only needs an IEnumerable<T> and not a List<T> , because I just want to enumerate the items and I don't want to change the list.
Adding a list = list.ToList(); at the beginning of the method makes the warning go away:

public void DoSomething(IEnumerable<string> list)
{
    list = list.ToList();
    if (list.Any())
    {
        // ...
    }
    foreach (var item in list)
    {
        // ...
    }
}

I understand, why that makes the warning go away, but it looks a bit like a hack to me...
Any suggestions, how to solve that warning better and still use the most general type possible in the method signature?
The following problems should all be solved for a good solution:

1. No call to ToList() inside the method, because it has a performance impact
2. No usage of ICollection<T> or even more specialized interfaces/classes, because they change the semantics of the method as seen from the caller.
3. No multiple iterations over an IEnumerable<T> and thus risking accessing a database multiple times or similar.

Note: I am aware that this is not a Resharper issue, and thus, I don't want to suppress this warning, but fix the underlying cause as the warning is legit.

UPDATE: Please don't care about Any and the foreach . I don't need help in merging those statements to have only one enumeration of the enumerable.
It could really be anything in this method that enumerates the enumerable multiple times!

Answer 1

This class will give you a way to split the first item off of the enumeration and then have an IEnumerable for the rest of the enumeration without giving you a double enumeration, thus avoiding the potentially nasty performance hit. It's usage is like this (where T is whatever type you are enumerating):

var split = new SplitFirstEnumerable(currentIEnumerable);
T firstItem = split.First;
IEnumerable<T> remaining = split.Remaining;

Here is the class itself:

/// <summary>
/// Use this class when you want to pull the first item off of an IEnumerable
/// and then enumerate over the remaining elements and you want to avoid the
/// warning about "possible double iteration of IEnumerable" AND without constructing
/// a list or other duplicate data structure of the enumerable. You construct 
/// this class from your existing IEnumerable and then use its First and 
/// Remaining properties for your algorithm.
/// </summary>
/// <typeparam name="T">The type of item you are iterating over; there are no
/// "where" restrictions on this type.</typeparam>
public class SplitFirstEnumerable<T>
{
    private readonly IEnumerator<T> _enumerator;

    /// <summary>
    /// Constructor
    /// </summary>
    /// <remarks>Will throw an exception if there are zero items in enumerable or 
    /// if the enumerable is already advanced past the last element.</remarks>
    /// <param name="enumerable">The enumerable that you want to split</param>
    public SplitFirstEnumerable(IEnumerable<T> enumerable)
    {
        _enumerator = enumerable.GetEnumerator();
        if (_enumerator.MoveNext())
        {
            First = _enumerator.Current;
        }
        else
        {
            throw new ArgumentException("Parameter 'enumerable' must have at least 1 element to be split.");
        }
    }

    /// <summary>
    /// The first item of the original enumeration, equivalent to calling
    /// enumerable.First().
    /// </summary>
    public T First { get; private set; }

    /// <summary>
    /// The items of the original enumeration minus the first, equivalent to calling
    /// enumerable.Skip(1).
    /// </summary>
    public IEnumerable<T> Remaining
    {
        get
        {
            while (_enumerator.MoveNext())
            {
                yield return _enumerator.Current;
            }
        }
    }
}

This does presuppose that the IEnumerable has at least one element to start. If you want to do more of a FirstOrDefault type setup, you'll need to catch the exception that would otherwise be thrown in the constructor.

Answer 2

There exists a general solution to address both Resharper warnings: the lack of guarantee for repeat-ability of IEnumerable, and the List base class (or potentially expensive ToList() workaround).

Create a specialized class, IE "RepeatableEnumerable", implementing IEnumerable, with "GetEnumerator()" implemented with the following logic outline:

1. Yield all items already collected so far from the inner list.

2. If the wrapped enumerator has more items,

   • While the wrapped enumerator can move to the next item,

     1. Get the current item from the inner enumerator.

     2. Add the current item to the inner list.

     3. Yield the current item

3. Mark the inner enumerator as having no more items.

Add extension methods and appropriate optimizations where the wrapped parameter is already repeatable. Resharper will no longer flag the indicated warnings on the following code:

public void DoSomething(IEnumerable<string> list)
{
    var repeatable = list.ToRepeatableEnumeration();
    if (repeatable.Any()) // <- no warning here anymore.
      // Further, this will read at most one item from list.  A
      // query (SQL LINQ) with a 10,000 items, returning one item per second
      // will pass this block in 1 second, unlike the ToList() solution / hack.
    {
        // ...
    }

    foreach (var item in repeatable) // <- and no warning here anymore, either.
      // Further, this will read in lazy fashion.  In the 10,000 item, one 
      // per second, query scenario, this loop will process the first item immediately
      // (because it was read already for Any() above), and then proceed to
      // process one item every second.
    {
        // ...
    }
}

With a little work, you can also turn RepeatableEnumerable into LazyList, a full implementation of IList. That's beyond the scope of this particular problem though. :)

UPDATE: Code implementation requested in comments -- not sure why the original PDL wasn't enough, but in any case, the following faithfully implements the algorithm I suggested (My own implementation implements the full IList interface; that is a bit beyond the scope I want to release here... :) )

public class RepeatableEnumerable<T> : IEnumerable<T>
{
    readonly List<T> innerList;
    IEnumerator<T> innerEnumerator;

    public RepeatableEnumerable( IEnumerator<T> innerEnumerator )
    {
        this.innerList = new List<T>();
        this.innerEnumerator = innerEnumerator;
    }

    public IEnumerator<T> GetEnumerator()
    {
        // 1. Yield all items already collected so far from the inner list.
        foreach( var item in innerList ) yield return item;

        // 2. If the wrapped enumerator has more items
        if( innerEnumerator != null )
        {
            // 2A. while the wrapped enumerator can move to the next item
            while( innerEnumerator.MoveNext() )
            {
                // 1. Get the current item from the inner enumerator.
                var item = innerEnumerator.Current;
                // 2. Add the current item to the inner list.
                innerList.Add( item );
                // 3. Yield the current item
                yield return item;
            }

            // 3. Mark the inner enumerator as having no more items.
            innerEnumerator.Dispose();
            innerEnumerator = null;
        }
    }

    System.Collections.IEnumerator System.Collections.IEnumerable.GetEnumerator()
    {
        return GetEnumerator();
    }
}

// Add extension methods and appropriate optimizations where the wrapped parameter is already repeatable.
public static class RepeatableEnumerableExtensions
{
    public static RepeatableEnumerable<T> ToRepeatableEnumerable<T>( this IEnumerable<T> items )
    {
        var result = ( items as RepeatableEnumerable<T> )
            ?? new RepeatableEnumerable<T>( items.GetEnumerator() );
        return result;
    }
}

Answer 3

You should probably take an IEnumerable<T> and ignore the "multiple iterations" warning.

This message is warning you that if you pass a lazy enumerable (such as an iterator or a costly LINQ query) to your method, parts of the iterator will execute twice.

Answer 4

There is no perfect solution, choose one acording to the situation.

• enumerable.ToList, you may optimize it by firstly trying "enumerable as List" as long as you don't modify the list
• Iterate two times over the IEnumerable but make it clear for the caller (document it)
• Split in two methods
• Take List to avoid cost of "as"/ToList and potential cost of double enumeration

The first solution (ToList) is probably the most "correct" for a public method that could be working on any Enumerable.

You can ignore Resharper issues, the warning is legit in a general case but may be wrong in your specific situation. Especially if the method is intended for internal usage and you have full control on callers.

Answer 5

I realize this question is old and already marked as answered, but I was surprised that nobody suggested manually iterating over the enumerator:

// NOTE: list is of type IEnumerable<T>.
//       The name was taken from the OP's code.
var enumerator = list.GetEnumerator();
if (enumerator.MoveNext())
{
    // Run your list.Any() logic here
    ...

    do
    {
        var item = enumerator.Current;
        // Run your foreach (var item in list) logic here
        ...
    } while (enumerator.MoveNext());
}

Seems a lot more straightforward than the other answers here.

Answer 6

UIMS* - Fundamentally, there is no great solve. IEnumerable<T> used to be the "very basic thing that represents a bunch of things of the same type, so using it in method sigs is Correct." It has now also become a "thing that might evaluate behind the scenes, and might take a while, so now you always have to worry about that."

It's as if IDictionary suddenly were extended to support lazy loading of values, via a LazyLoader property of type Func<TKey,TValue>. Actually that'd be neat to have, but not so neat to be added to IDictionary, because now every time we receive an IDictionary we have to worry about that. But that's where we are.

So it would seem that "if a method takes an IEnumerable and evals it twice, always force eval via ToList()" is the best you can do. And nice work by Jetbrains to give us this warning.

*(Unless I'm Missing Something. . . just made it up but it seems useful)

Answer 7

Generally speaking, what you need is some state object into which you can PUSH the items (within a foreach loop), and out of which you then get your final result.

The downside of the enumerable LINQ operators is that they actively enumerate the source instead of accepting items being pushed to them, so they don't meet your requirements.

If you eg just need the minimum and maximum values of a sequence of 1'000'000 integers which cost $1'000 worth of processor time to retrieve, you end up writing something like this:

public class MinMaxAggregator
{
    private bool _any;
    private int _min;
    private int _max;

    public void OnNext(int value)
    {
        if (!_any)
        {
            _min = _max = value;
            _any = true;
        }
        else
        {
            if (value < _min) _min = value;
            if (value > _max) _max = value;
        }
    }

    public MinMax GetResult()
    {
        if (!_any) throw new InvalidOperationException("Sequence contains no elements.");
        return new MinMax(_min, _max);
    }
}

public static MinMax DoSomething(IEnumerable<int> source)
{
    var aggr = new MinMaxAggregator();
    foreach (var item in source) aggr.OnNext(item);
    return aggr.GetResult();
}

In fact, you just re-implemented the logic of the Min() and Max() operators. Of course that's easy, but they are only examples for arbitrary complex logic you might otherwise easily express in a LINQish way.

The solution came to me on yesterday's night walk: we need to PUSH... that's REACTIVE. All the beloved operators also exist in a reactive version built for the push paradigm, They can be chained together at will to whatever complexity you need. just as their enumerable counterparts.

So the min/max example boils down to:

public static MinMax DoSomething(IEnumerable<int> source)
{
    // bridge over to the observable world
    var connectable = source.ToObservable(Scheduler.Immediate).Publish();
    // express the desired result there (note: connectable is observed by multiple observers)
    var combined = connectable.Min().CombineLatest(connectable.Max(), (min, max) => new MinMax(min, max));
    // subscribe
    var resultAsync = combined.GetAwaiter();
    // unload the enumerable into connectable
    connectable.Connect();
    // pick up the result
    return resultAsync.GetResult();
}

Answer 8

Why not:

bool any;

foreach (var item in list)
{
    any = true;
    // ...
}
if(any)
{
    //...
}

Update: Personally, I wouldn't drastically change the code just to get around a warning like this. I would just disable the warning and continue on. The warning is suggesting you change the general flow of the code to make it better; if you're not making the code better (and arguably making it worse) to address the warning; then the point of the warning is missed.

For example:

// ReSharper disable PossibleMultipleEnumeration
        public void DoSomething(IEnumerable<string> list)
        {
            if (list.Any()) // <- here
            {
                // ...
            }
            foreach (var item in list) // <- and here
            {
                // ...
            }
        }
// ReSharper restore PossibleMultipleEnumeration

Answer 9

Be careful when accepting enumerables in your method. The "warning" for the base type is only a hint, the enumeration warning is a true warning.

However, your list will be enumerated at least two times because you do any and then a foreach. If you add a ToList() your enumeration will be enumerated three times - remove the ToList().

I would suggest to set resharpers warning settings for the base type to a hint. So you still have a hint (green underline) and the possibility to quickfix it (alt+enter) and no "warnings" in your file.

You should take care if enumerating the IEnumerable is an expensive action like loading something from file or database, or if you have a method which calculates values and uses yield return. In this case do a ToList() or ToArray() first to load/calculate all data only ONCE.

Answer 10

You could use ICollection<T> (or IList<T> ). It's less specific than List<T> , but doesn't suffer from the multiple-enumeration problem.

Still I'd tend to use IEnumerable<T> in this case. You can also consider to refactor the code to enumerate only once.

Answer 11

Use an IList as your parameter type rather than IEnumerable - IEnumerable has different semantics to List whereas IList has the same

IEnumerable could be based on a non-seekable stream which is why you get the warnings

Answer 12

You can iterate only once:

public void DoSomething(IEnumerable<string> list)
{
    bool isFirstItem = true;
    foreach (var item in list)
    {
        if (isFirstItem)
        {
            isFirstItem = false;
            // ...
        }
        // ...
    }
}

Answer 13

There is something no one had said before (@Zebi). Any() already iterates trying to find the element. If you call a ToList(), it will iterate as well, to create a list. The initial idea of using IEnumerable is only to iterate, anything else provokes an iteration in order to perform. You should try to, inside a single loop, do everything.

And include in it your.Any() method.

if you pass a list of Action in your method you would have a cleaner iterated once code

public void DoSomething(IEnumerable<string> list, params Action<string>[] actions)
{
    foreach (var item in list)
    {
        for(int i =0; i < actions.Count; i++)
        {
           actions[i](item);
        }
    }
}