How to leverage dynamic programming inside of a Linq Where clause?

Question

I'm trying to find a way to pass or use a collection inside of a .Where() clause to speed up the execution, but haven't had any luck. Here's what I'd like to do:

var matches = superLongEnumerable.Where((x, HashSet<string> dynamicSet) => 
    {
        var parent = SemiExpensiveCallToGetParent(x);
        if(dynamicSet.Contains(parent))
        {
            // DP optimization to save further computation
            return true;
        }
        var matched = ExpensiveCallToCheckMatch(parent);
        if(matched) {
            dynamicSet.Add(parent);
        }
        return matched;
    });

As far as I can tell, .Where() only supports the current element, and optionally the index of the current element. Is there an alternate to .Where() that I can use?

Answer 1

If it is visible in the same scope as your matches variable, you can use it without having to pass it in. Provided there's a dynamicSet variable defined outside of you call to Where() , just use Where((x) => ...) . You don't need to pass it in.

Answer 2

Since this is LINQ to entities obviously, you can always you a simple for or foreach for simple scenarios, or you build up an enumerator function.

But the main problem for performance is: you only save the result of your expensive test, in case of success, unsuccessful expensive calls you repeat again and again. So a dictionary might be more useful.

An alternative to Where would be

 public IEnumerable<string> MyFilter(IEnumerable<string> source)
 {
      var temp = new Dictionary<string, bool>();
      foreach(var item in source)
      {
           var parent = SemiExpensiveCallToGetParent(item);
           if (temp.TryGetValue(parent, out bool result))
           {
               if (result)
                    yield return item;
           }

           var matched = ExpensiveCallToCheckMatch(parent);
           temp.Add(parent, matched);
           if (matched)
               yield return item;
      }
  }

If you like it with WHERE you can use a class method:

  public class Helper
  {
       private readonly Dictionary<string,bool> tmp = new Dictionary<string, bool>();

       public bool Condition(string item)
       {
           var parent = SemiExpensiveCallToGetParent(item);
           if (temp.TryGetValue(parent, out bool result))
              return result;
           var matched = ExpensiveCallToCheckMatch(parent);
           temp.Add(parent, matched);
           return matched;
       }
    }

and this you call with

    yourCollection.Where(new Helper().Condition);

To be underlined once again: It's LINQ to entities only, not for LINQ to SQL.

And put your prefered StringComparer in the constructor call of the Dictionary.