[英]Azure Blob storage .Net client library and TCP connections over time
We are using Azure blob storage to store incoming messages from a data feed (we receive about 1.5GB a day of data) and then processing them which is triggered via a message queue (rabbitmq). 我们正在使用Azure blob存储来存储来自数据源的传入消息(我们每天接收大约1.5GB的数据),然后处理通过消息队列(rabbitmq)触发的消息。 Here is how the setup looks like:
设置如下所示:
Producer -> Store XML file in Azure blob -> Publish blob address to queue 生产者->在Azure blob中存储XML文件->将blob地址发布到队列
Consumer -> Read the blob address from the queue -> Download blob in memory 使用者->从队列中读取Blob地址->在内存中下载Blob
And here is the Download blob method that is executed for each message: 这是对每条消息执行的Download blob方法:
private string GetBlobText(string containerName, string blobName)
{
// Parse the connection string and return a reference to the storage account.
CloudStorageAccount storageAccount = CloudStorageAccount.Parse(Settings.Default.DatafeedStorageConnString);
CloudBlobClient blobClient = storageAccount.CreateCloudBlobClient();
CloudBlobContainer container = blobClient.GetContainerReference(containerName);
CloudBlockBlob blockBlob = container.GetBlockBlobReference(blobName);
return blockBlob.DownloadText(Encoding.UTF8);
}
This pipeline is running at a fairly frequent pace and hence we see that over time (some weeks) the program starts receiving Socket errors. 该管道以相当频繁的速度运行,因此,随着时间的推移(几周),我们看到程序开始接收Socket错误。 Here is the trace:
这是跟踪:
Microsoft.WindowsAzure.Storage.StorageException: Unable to connect to the remote server ---> System.Net.WebException: Unable to connect to the remote server ---> System.Net.Sockets.SocketException: An operation on a socket could not be performed because the system lacked sufficient buffer space or because a queue was full 40.68.232.24:443
at System.Net.Sockets.Socket.DoConnect(EndPoint endPointSnapshot, SocketAddress socketAddress)
at System.Net.ServicePoint.ConnectSocketInternal(Boolean connectFailure, Socket s4, Socket s6, Socket& socket, IPAddress& address, ConnectSocketState state, IAsyncResult asyncResult, Exception& exception)
--- End of inner exception stack trace ---
at System.Net.HttpWebRequest.GetResponse()
at Microsoft.WindowsAzure.Storage.Core.Executor.Executor.ExecuteSync[T](RESTCommand`1 cmd, IRetryPolicy policy, OperationContext operationContext) in c:\Program Files (x86)\Jenkins\workspace\release_dotnet_master\Lib\ClassLibraryCommon\Core\Executor\Executor.cs:line 695
--- End of inner exception stack trace ---
at Microsoft.WindowsAzure.Storage.Core.Executor.Executor.ExecuteSync[T](RESTCommand`1 cmd, IRetryPolicy policy, OperationContext operationContext) in c:\Program Files (x86)\Jenkins\workspace\release_dotnet_master\Lib\ClassLibraryCommon\Core\Executor\Executor.cs:line 604
at Microsoft.WindowsAzure.Storage.Blob.CloudBlob.DownloadRangeToStream(Stream target, Nullable`1 offset, Nullable`1 length, AccessCondition accessCondition, BlobRequestOptions options, OperationContext operationContext) in c:\Program Files (x86)\Jenkins\workspace\release_dotnet_master\Lib\ClassLibraryCommon\Blob\CloudBlob.cs:line 675
at Microsoft.WindowsAzure.Storage.Blob.CloudBlob.DownloadToStream(Stream target, AccessCondition accessCondition, BlobRequestOptions options, OperationContext operationContext) in c:\Program Files (x86)\Jenkins\workspace\release_dotnet_master\Lib\ClassLibraryCommon\Blob\CloudBlob.cs:line 234
at Microsoft.WindowsAzure.Storage.Blob.CloudBlockBlob.DownloadText(Encoding encoding, AccessCondition accessCondition, BlobRequestOptions options, OperationContext operationContext) in c:\Program Files (x86)\Jenkins\workspace\release_dotnet_master\Lib\ClassLibraryCommon\Blob\CloudBlockBlob.cs:line 1279
This seems like an issue with not handling connections efficiently, we had tried reusing blobClient, but that has not helped us. 这似乎是无法有效处理连接的问题,我们曾尝试过重用blobClient,但这并没有帮助我们。 What shall we look into further to solve this?
我们应该进一步研究以解决这个问题吗?
Microsoft.WindowsAzure.Storage.StorageException: Unable to connect to the remote server ---> System.Net.WebException: Unable to connect to the remote server ---> System.Net.Sockets.SocketException: An operation on a socket could not be performed because the system lacked sufficient buffer space or because a queue was full 40.68.232.24:443 (blob.am5prdstr02a.store.core.windows.net)
Microsoft.WindowsAzure.Storage.StorageException:无法连接到远程服务器---> System.Net.WebException:无法连接到远程服务器---> System.Net.Sockets.SocketException:对套接字的操作可能由于系统缺少足够的缓冲区空间或队列已满40.68.232.24:443(blob.am5prdstr02a.store.core.windows.net),因此无法执行
Per my understanding, it may be TCP/IP Port Exhaustion. 据我了解,可能是TCP / IP端口耗尽。 I assumed that you could use Netstat to query the state of your network connections and the ports you are using.
我假设您可以使用Netstat来查询网络连接的状态以及正在使用的端口。 For more details, you could refer to here for explanation and ways to diagnose this issue.
有关更多详细信息,您可以参考此处以获取解释和诊断此问题的方法。
If the Port Exhaustion occurs in your client computer, I assume you could increase the the upper range of ephemeral ports for client TCP/IP socket connections and reduce the client TCP/IP socket connection timeout, for more details, you could refer to here . 如果客户端计算机中发生端口耗尽,我假定您可以增加客户端TCP / IP套接字连接的临时端口上限,并减少客户端TCP / IP套接字连接超时,有关更多详细信息,请参阅此处 。 Also, you could scale your client application into multiple instances among different computers.
同样,您可以将客户端应用程序扩展到不同计算机之间的多个实例中。
声明:本站的技术帖子网页,遵循CC BY-SA 4.0协议,如果您需要转载,请注明本站网址或者原文地址。任何问题请咨询:yoyou2525@163.com.