How to add items to a collection, while filtering it?

Question

I have a collection of log events. I give the user the ability to filter out different log levels. During that filtering process, if a new log event is added, an exception is thrown. I can just lock the collection while filtering it, however this means that any new log events won't be added in the correct order, or at all(?).

The filtering method:

//Filters and returns a list of filtered log events
    private IEnumerable<LogEvent> FilterLogEvents()
    {
        return (from x in LogEvents
                where ((ViewDebugLogs == true) ? x.Level == "Debug" : false)
                || ((ViewErrorLogs == true) ? x.Level == "Error" : false)
                || ((ViewInfoLogs == true) ? x.Level == "Info" : false)
                select x);
    }

If I just return a list, the same thing happens if it's in mid-query.

How do I go about adding to a collection, while at the same time using it? The main concern here is making sure log events DO get added in the proper order to the collection if it is added mid-filter. So if I cannot add the log events at the same time, can I somehow queue them up and insert them correctly afterwards?

Answer 1

No, a collection can't be modified when being enumerated. You can copy the collection, enumerate the copy, and modify the original collection during the enumeration.

//the following example throws an exception
var numbers = GetTheNumbers();
foreach(var number in numbers)
{
    numbers.Add(1);   //exception thrown
}

//copy the collection first
var numbers = GetTheNumbers();
var copy = numbers.ToArray();
foreach(var number in copy)
{
    numbers.Add(1);  //safe
}

Answer 2

You can iterate over some of the concurrent collections while adding items to them. One such collection is ConcurrentQueue<T> , which I think might be suitable for a log if the following conditions are true:

• You only want to add items at the end of the log.
• You never need to empty it in one go. (Rather than emptying a queue, you'd have to create a new empty one, or you'd have to loop calling TryDequeue() until it was empty).

Note that when you iterate over a ConcurrentQueue<T> it appears that the count is read at the start of the iteration and not updated during the iteration, so if new items are added while you're iterating you won't see them (but the addition of the items won't cause an exception).

Here's a demo program:

using System;
using System.Collections.Concurrent;
using System.Threading;
using System.Threading.Tasks;

namespace Demo
{
    internal class Program
    {
        public static void Main()
        {
            var queue = new ConcurrentQueue<int>();

            Task.Run(() =>
            {
                for (int i = 0; i < 1000; ++i)
                {
                    queue.Enqueue(i);
                    Thread.Sleep(1);
                }
            });

            Thread.Sleep(100);
            Console.WriteLine($"Count before loop: {queue.Count}");

            foreach (var i in queue)
                Console.WriteLine($"{i}, n={queue.Count}");
        }
    }
}