將偵聽器事件添加到 ConcurrentQueue 或 ConcurrentBag？

Question

我有多個任務以 1:1 的方式從隊列中獲取消息。 我想將來自每個線程的這些消息添加到 ConcurrentBag 中，並在它們異步進入時對其進行處理。 此處的目的是盡快將消息從隊列中取出，以免隊列填滿。 我只需要一個監聽器的幫助，等待消息添加到 ConcurrentBag 然后我需要從包中刪除消息並處理它們

private static ConcurrentQueue<string> messageList = new ConcurrentQueue<string>();
private static readonly SemaphoreSlim semaphore = new SemaphoreSlim(50);
void Main (string[] args)
{
   List<Task> taskList = new TaskList();
    foreach(var job in JobList)
   {
      taskList.Add(Task.Run(() => ListenToQueue(job.QueueName));
   }
  Task.WaitAll(taskList.ToArray());
}


private async Task<string> ListenToQueue(string queueName)
{
   var cancellationtoken = new CancellationTokenSource(TimeSpan.FromMinutes(60)).Token
  
   //possibly 5000 messages can be on a single queue 
    while(!cancellationtoken.IsCancellationRequested)
     {
      var message = getMessageFromQueue(queueName);
      messageList.Enqueue(message); //Add the message from each thread to a thread safe List
     }
}

我需要一個偵聽器事件，每次將某些內容添加到列表中時，都會觸發此事件。 我還需要以線程安全的方式從列表中刪除消息。

 private void Listener()
   {
      var msg =  string.Empty;
       while (messageList.Count > 0)
     {
         messageList.TryDequeue(out msg)
         await semaphore.WaitAsync();
            Task.Run(() => 
                  { 
                    try
                    {
                      if(!String.IsNullorEmpty(msg))
                      {
                     _ = ProcessMessage(msg); // I do not want to await anything but just fire and let it go
                       }
                    }
                    finally
                    {
                       sim.Release();
                    }
                  });
      }    
    }

Answer 1

這些天來，我建議使用像System.Threading.Channels這樣的異步兼容解決方案：

private static Channel<string> messageList = Channel.CreateUnbounded<string>();

private async Task<string> ListenToQueue(string queueName)
{
  var cancellationtoken = new CancellationTokenSource(TimeSpan.FromMinutes(60)).Token;

  try
  {
    var message = await getMessageFromQueue(queueName, cancellationtoken);
    await messageList.Writer.WriteAsync(message, cancellationtoken);
  }
  catch (OperationCanceledException)
  {
    // ignored
  }
}

private async Task Listener()
{
  await foreach (var msg in messageList.Reader.ReadAllAsync())
  {
    if (!string.IsNullOrEmpty(msg))
      _ = Task.Run(() => ProcessMessage(msg));
  }
}

但是，如果您想（或需要）留在阻塞的世界中，那里也有一個解決方案。 ConcurrentBag<T>和ConcurrentQueue<T>很少直接使用。 相反，更常見的是使用BlockingCollection<T> ，它包裝並發集合並提供更高級別的 API，包括GetConsumingEnumerable ：

private static BlockingCollection<string> messageList = new();

private async Task<string> ListenToQueue(string queueName)
{
  var cancellationtoken = new CancellationTokenSource(TimeSpan.FromMinutes(60)).Token;

  try
  {
    var message = await getMessageFromQueue(queueName, cancellationtoken);
    messageList.Add(message, cancellationtoken);
  }
  catch (OperationCanceledException)
  {
    // ignored
  }
}

private void Listener()
{
  foreach (var msg in messageList.GetConsumingEnumerable())
  {
    if (!string.IsNullOrEmpty(msg))
      _ = Task.Run(() => ProcessMessage(msg));
  }
}