Multi-threading in Enumerator

Question

I am using C# and I have an enumerator and I am reading the data inside the enumerator sequentially.

This is a third party library object and does not support Parallel.Foreach

while(enumerator.Next())
{
  var item = enumerator.Read();
  ProcessItem(item);
}

ProcessItem(Item item)
{
// Is lock required here
 if(item.prop == "somevalue")
   this._list.Add(item);
}

I want to achieve multithreading here while reading the content.

while(enumerator.Next())
{
  // This code should run in a multi-threaded way

  var item = enumerator.Read();
  // ProcessItem method puts these items on a class level list property 
  // Is there any Lock required?
  ProcessItem(item);
}

I am new to multithreading. Please share any code samples which satisfies the above requirement.

Answer 1

Yes, some locking required. you can achieve it using lock or using a concurrent collection type.

using lock :

ProcessItem(Item item)
{
    if(item.prop == "somevalue")
    {
        lock(_list)
        {
            _list.Add(item);
        }
    }
}

Edit: based on detail you provided, you can wrap the enumerator from external lib using your own enumerator like below so you can use Parallel.ForEach on it:

We assume the enumerator you got is something like MockEnumerator , we wrap it in a normal IEnumerator, and IEnumerable so we are able to use Parallel.ForEach to read in parallel.

class Program
{
    class Item
    {
        public int SomeProperty { get; }

        public Item(int prop)
        {
            SomeProperty = prop;
        }
    }

    class MockEnumerator
    {
        private Item[] _items = new Item[] { new Item(1), new Item(2) };
        private int _position = 0;

        public bool Next()
        {
            return _position++ < _items.Length;
        }

        public Item Read()
        {
            return _items[_position];
        }
    }

    class EnumeratorWrapper : IEnumerator<Item>, IEnumerable<Item>
    {
        private readonly MockEnumerator _enumerator;

        public EnumeratorWrapper(MockEnumerator enumerator)
        {
            this._enumerator = enumerator;
        }

        public Item Current => _enumerator.Read();

        object IEnumerator.Current => Current;

        public void Dispose()
        {
        }

        public IEnumerator<Item> GetEnumerator()
        {
            throw new NotImplementedException();
        }

        public bool MoveNext()
        {
            return _enumerator.Next();
        }

        public void Reset()
        {
        }

        IEnumerator IEnumerable.GetEnumerator()
        {
            return this;
        }
    }

    private static List<Item> _list = new List<Item>();

    static void Main(string[] args)
    {
        var enumerator = new EnumeratorWrapper(new MockEnumerator());
        Parallel.ForEach(enumerator, item =>
        {
            if (item.SomeProperty == 1)//someval
            {
                lock (_list)
                {
                    _list.Add(item);
                }
            }
        });
    }
}

Answer 2

This is a good example for task-based parallelization. Each processing of an item corresponds to a task. Hence, you can change the loop to the following:

  var tasks = new List<Task<int>>();
  while(enumerator.MoveNext())
  {
    var item = enumerator.Current;

    Task<int> task = new Task<int>(() => ProcessItem(item));
    task.Start();
    tasks.Add(task);
  }


  foreach(Task<int> task in tasks)
  {
    int i = task.Result;
    classList.Add(i);
  }

Note that the synchronization on the classList is implicitly given by first spawning all tasks in the while loop and then merging the results in the foreach loop. The synchronization is specifically given by the access to Result which waits until the corresponding task is finished .