简体   繁体   English

当应用于Azure存储库ListBlobs()的结果时,是否有办法使foreach循环的MoveNext()部分重试该操作?

[英]Is there a way to make the MoveNext() part of foreach loop retry the operation when applied to result of Azure Storage Library ListBlobs()?

I have code using Azure Storage Library that goes like this: 我有使用Azure存储库的代码,如下所示:

CloudBlobContainer container = obtainContainer();

var blobList = container.ListBlobs(options:OptionsWithLinearRetry);
foreach (var blobItem in blobList) // << exception happens here sometimes
{
     //process item
}

and it works okay most of the time. 大部分时间都可以。 But sometimes something goes wrong with the network and then I get: 但是有时网络出现问题,然后我得到:

Microsoft.WindowsAzure.Storage.StorageException: Unable to read data from the transport connection: An existing connection was forcibly closed by the remote host. Microsoft.WindowsAzure.Storage.StorageException:无法从传输连接读取数据:远程主机强行关闭了现有连接。 ---> System.IO.IOException: Unable to read data from the transport connection: An existing connection was forcibly closed by the remote host. ---> System.IO.IOException:无法从传输连接中读取数据:现有连接被远程主机强行关闭。 ---> System.Net.Sockets.SocketException: An existing connection was forcibly closed by the remote host ---> System.Net.Sockets.SocketException:现有的连接被远程主机强行关闭

and the stack is like this: 和堆栈是这样的:

// lots of Azure Storage Library stuff here
 at Microsoft.WindowsAzure.Storage.Blob.CloudBlobContainer.<>c__DisplayClass13.<ListBlobs>b__12(IContinuationToken token)
at Microsoft.WindowsAzure.Storage.Core.Util.General.<LazyEnumerable>d__0`1.MoveNext()
// my code with foreach line here

and so it looks like the foreach causes a MoveNext() call and the latter fails when obtaining the new chunk of data. 因此,看起来foreach引起了MoveNext()调用,而后者在获取新的数据块时失败。

Can I somehow cause that MoveNext() call to be retried? 我可以以某种方式导致重试MoveNext()调用吗? Is there a way to have a "try-catch-retry" logic in foreach loop? 有没有办法在foreach循环中使用“ try-catch-retry”逻辑?

No matter how retry in next milliseconds, it will throw same exception, until it's totally closed. 无论在接下来的几毫秒内如何重试,它都会抛出相同的异常,直到完全关闭为止。 because your storage account is hitting the thresholds/limit which is specified by Windows Azure Storage: https://blogs.msdn.microsoft.com/windowsazurestorage/2012/11/04/cross-post-windows-azures-flat-network-storage-and-2012-scalability-targets/ 因为您的存储帐户达到Windows Azure存储所指定的阈值/限制https : //blogs.msdn.microsoft.com/windowsazurestorage/2012/11/04/cross-post-windows-azures-flat-network-存储和2012年可扩展性目标/

Actually not only Azure, also Google services and AWS are same, all of them have thresholds on remote calls. 实际上,不仅Azure,Google服务和AWS都是相同的,它们都具有远程调用的阈值。

The MSDN cloudblobcontainer.listblobs says: MSDN cloudblobcontainer.listblobs说:

Returns an enumerable collection of the blobs in the container that are retrieved lazily . 返回容器中延迟检索的Blob的可枚举集合。

you can setup 你可以设置

  1. BlobRequestOptions.RetryPolicy in next 0-30 seconds. BlobRequestOptions.RetryPolicy在接下来的0-30秒内。
  2. BlobRequestOptions.AbsorbConditionalErrorsOnRetry BlobRequestOptions.AbsorbConditionalErrorsOnRetry
  3. BlobRequestOptions.MaximumExecutionTime BlobRequestOptions.MaximumExecutionTime

You detect retry by the events: 您可以通过事件检测重试:

  1. OperationContext.GlobalRetrying OperationContext.GlobalRetrying
  2. OperationContext.Retrying OperationContext.Retrying

You could also try from Linq: 您也可以从Linq尝试:

  1. .ToList()
  2. .Where(x=>x.Conditions).ToList() filter useless data in server side. .Where(x=>x.Conditions).ToList()在服务器端过滤无用的数据。
  3. .Take(10) to page your result, because no one need all results in 1 page. .Take(10)分页结果,因为没有人需要一页中的所有结果。

声明:本站的技术帖子网页,遵循CC BY-SA 4.0协议,如果您需要转载,请注明本站网址或者原文地址。任何问题请咨询:yoyou2525@163.com.

 
粤ICP备18138465号  © 2020-2024 STACKOOM.COM