简体   繁体   English

在C#中从Feeder接收大量tcp套接字财务数据的最佳方法?

[英]Best way to receive high volume tcp socket financial data from a feeder in c#?

I am developing a C# Windows Service that will receive financial quotations from a feeder that uses TCP. 我正在开发C#Windows服务,它将从使用TCP的馈送器接收财务报价。 My project must receive and process a large volume of data, for I will be tracking 140 different assets that are used to update an SQL database every second. 我的项目必须接收和处理大量数据,因为我将跟踪140个用于每秒更新SQL数据库的不同资产。

I am using a looping to pool data from the socket, in a BackgroundWork thread: 我正在使用循环从BackgroundWork线程中的套接字中收集数据:

  try
  {
    // Send START command with the assets.
    if (!AeSocket.AeSocket.Send(Encoding.ASCII.GetBytes(String.Format("{0}{1}START{1}BC|ATIVO{1}{2}{3}", GlobalData.GlobalData.Id, GlobalData.GlobalData.Tab, AeSocket.AeSocket.GetAssets(),
                                                                      GlobalData.GlobalData.Ret))).Contains("OK"))
    {
      throw new Exception("Advise was no accepted.");
    }

    // Pool the socket and send all received string to the queue for processing in the background.
    while (true)
    {
      // Make sure the connection to the socket is still active.
      if (!AeSocket.AeSocket.Client.Connected)
      {
        throw new Exception("The connection was closed.");
      }

      // If no data is available in the socket, loop and keep waiting.
      if (!AeSocket.AeSocket.Client.Poll(-1, SelectMode.SelectRead))
      {
        continue;
      }

      // There are data waiting to be read.
      var data = new Byte[AeSocket.AeSocket.ReadBufferSize];
      var bytes = AeSocket.AeSocket.Client.Receive(data, 0);
      AeSocket.AeSocket.Response = Encoding.Default.GetString(data, 0, bytes);

      // Push into the queue for further processing in a different thread.
      GlobalData.GlobalData.RxQueue.Add(AeSocket.AeSocket.Response);
    }
  }
  catch
  {
    backgroundWorkerMain.CancelAsync();
  }
  finally
  {
    AeSocket.AeSocket.Client.Close();
    AeSocket.AeSocket.Client.Dispose();
  }

The received data is being processed in a separated thread to avoid blocking the socket receiving activity, due to the high data volume received. 接收到的数据在单独的线程中进行处理,以避免由于接收到的大量数据而阻塞套接字接收活动。 I am using a BlockingCollection (RxQueue). 我正在使用BlockingCollection(RxQueue)。

This collection is being observed as shown in the following code snippet: 如以下代码片段所示,正在观察该集合:

  // Subscribe to the queue for string processing in another thread.
  // This is a blocking observable queue, so it is run in this background worker thread.
  GlobalData.GlobalData.Disposable = GlobalData.GlobalData.RxQueue
    .GetConsumingEnumerable()
    .ToObservable()
    .Subscribe(c =>
    {
      try
      {
        ProcessCommand(c);
      }
      catch
      {
        // Any error will stop the processing.
        backgroundWorkerMain.CancelAsync();
      }
    });

The data is then added to a ConcurrentDictionary, to be read asynchronously by an one sec timer and saved to an SQL database: 然后将数据添加到ConcurrentDictionary中,以一秒钟的计时器异步读取并保存到SQL数据库中:

      // Add or update the quotation record in the dictionary.
      GlobalData.GlobalData.QuotDict.AddOrUpdate(dataArray[0], quot, (k, v) => quot);

The one second system timer is processed in another BackgroundWorker thread and it saves the quotations data read from the ConcurrentDictionary. 一秒钟的系统计时器在另一个BackgroundWorker线程中处理,并且保存从ConcurrentDictionary中读取的报价数据。

  // Update the gridViewAssetQuotations.
  var list = new BindingList<MyAssetRecord>();

  foreach (var kv in GlobalData.GlobalData.QuotDict)
  {
    // Add the data in a database table...
  }

Is this a good approach to this situation? 这是解决这种情况的好方法吗?

Using a BlockingCollection as an assynchronous queue and a ConcurrentDictionary to allow asynchronous reading from another thread is a good way of doing this? 使用BlockingCollection作为异步队列并使用ConcurrentDictionary允许从另一个线程异步读取是这样做的好方法吗?

What about the way I used to pool data from the socket: 那我以前从套接字中池数据的方式又如何呢:

      // If no data is available in the socket, loop and keep waiting.
      if (!AeSocket.AeSocket.Client.Poll(-1, SelectMode.SelectRead))
      {
        continue;
      }

Is there a better way of doing it? 有更好的方法吗?

Also, I have to send a KeepAlive command every 4 seconds to the TCP server. 另外,我必须每4秒向TCP服务器发送一个KeepAlive命令。 Could I do it totally asynchronously, ignoring the pooling loop above, using another system timer, or it must be synchronized to the pooling operation? 我可以使用另一个系统计时器来完全异步地忽略上述池循环,还是必须将其同步到池操作? The server only allows connection on one port. 服务器仅允许在一个端口上进行连接。

Thanks in advance for any suggestions. 在此先感谢您的任何建议。

Eduardo Quintana 爱德华多·金塔纳

Your loop will lead to high processor usage. 您的循环将导致处理器使用率升高。 Put a sleep (say 1 ms) in case there is no data to process, increase the sleep time if there is still no data in the next cycle, decrease the sleep time if you get data. 如果没有要处理的数据,请放置一个睡眠(例如1 ms),如果在下一个周期中仍然没有数据,则增加睡眠时间,如果获得数据,则减少睡眠时间。 That way the reader loop will auto adjust to data traffic and would save on processor cycles. 这样,读取器循环将自动适应数据流量,并节省处理器周期。 Rest all looks good. 休息都很好。

声明:本站的技术帖子网页,遵循CC BY-SA 4.0协议,如果您需要转载,请注明本站网址或者原文地址。任何问题请咨询:yoyou2525@163.com.

 
粤ICP备18138465号  © 2020-2024 STACKOOM.COM