Collection was modified error while processing data from socket stream

Question

I'm trying to capture tick level data from a Binance Aggregated data stream. As soon as the date changes, a function processes the data from the previous day. I'm using Task.Run to create another thread as new data for the current day is still streaming in and needs to be captured. However, I'm getting a 'Collection was modified enumeration operation may not execute.' in the foreach loop of the function that processes the data. I'm not sure why tradeData would be modified after it is passed to the processing function? Not sure if it matters but I'm capturing about 40 different symbols on separate threads.

private static void Start()
{
    foreach (SymbolData symbol in symbolData)
    {
         Task.Run(() => SubscribeToSymbol(symbol.symbol));
    }
}

private static void SubscribeToSymbol(string symbol)
{
    Dictionary<decimal, decimal> tradeData = new Dictionary<decimal, decimal>();
    DateTime lastTrade = DateTime.UtcNow;
    try
    {
         var socketClient = new BinanceSocketClient();
         socketClient.FuturesUsdt.SubscribeToAggregatedTradeUpdates(symbol, data =>
         {
              if (data.TradeTime.Date > lastTrade.Date)
              {
                   Task.Run(() => ProcessData(symbol, lastTrade.Date, tradeData));
                   tradeData.Clear();
              }
              lastTrade = data.TradeTime;

              if (tradeData.ContainsKey(data.Price))
              {
                   tradeData[data.Price] = tradeData[data.Price] + data.Quantity;
              }
              else
              {
                   tradeData[data.Price] = data.Quantity;
              }
         });
     }
     catch { }
}


private static void ProcessData(string symbol, DateTime date, Dictionary<decimal, decimal> tradeData)
{
     foreach (var price in tradeData)
     {
     //Error: Collection was modified enumeration operation may not execute.
     }
}

Answer 1

You're calling Task.Run repeatedly and as a result have more than one thread working with your tradeData object which is why you're seeing the collection modified exception.

I would recommend you rethink the code design so that you dont have multiple threads working with the same objects. If you absolutely must have threads working with the same objects then lock should be used:

private static object _objlock = new object();

private static void ProcessData(string symbol, DateTime date, Dictionary<decimal, decimal> tradeData)
{
    lock (_objlock)
    {
        foreach (var price in tradeData)
        {
     
        }
     }
}

Just be careful with deadlocks when using lock. I'd recommend reading up on locks and multi threading in C#.

Answer 2

By the time you start the foreach loop over the dictionary, the outside is "clearing" it. You should make a copy and send to ProcessData .

Task.Run(() => ProcessData(symbol, lastTrade.Date, tradeData.ToArray()));

And Receive the copy:

private static void ProcessData(string symbol, DateTime date, KeyValuePair<decimal, decimal>[] tradeData)
{
    foreach (var price in tradeData)
    {
// ... do your thing here
    }
}

Or, even more simple, just recreate a new dictionary instead of clearing it:

//tradeData.Clear();
tradeData = new Dictionary<decimal, decimal>();

Answer 3

Once time I had similar issue when I was removing value from list in foreach loop, I've changed it to simple for loop and that help you could try it.