[英]The specified blob does not exist on downloadAttributes with azure-storage-api java
We have application wherein many (~1000) consumers try to fetch files from blob storage. 我们有许多(〜1000)个使用者尝试从Blob存储中获取文件的应用程序。 There is no concurrent access on blob files, but they share single storage account.
Blob文件没有并发访问权限,但是它们共享一个存储帐户。 I see files available on the blob storage, but we are constantly seeing below exception
我看到Blob存储上有可用的文件,但是我们不断看到以下异常
Caused by: com.microsoft.azure.storage.StorageException: The specified blob does not exist.
at com.microsoft.azure.storage.StorageException.translateFromHttpStatus(StorageException.java:207)[3:org.ops4j.pax.logging.pax-logging-service:1.6.9]
at com.microsoft.azure.storage.StorageException.translateException(StorageException.java:172)[3:org.ops4j.pax.logging.pax-logging-service:1.6.9]
at com.microsoft.azure.storage.core.StorageRequest.materializeException(StorageRequest.java:306)[3:org.ops4j.pax.logging.pax-logging-service:1.6.9]
at com.microsoft.azure.storage.core.ExecutionEngine.executeWithRetry(ExecutionEngine.java:177)[3:org.ops4j.pax.logging.pax-logging-service:1.6.9]
at com.microsoft.azure.storage.blob.CloudBlob.downloadAttributes(CloudBlob.java:1268)[3:org.ops4j.pax.logging.pax-logging-service:1.6.9]
at com.microsoft.azure.storage.blob.CloudBlob.downloadAttributes(CloudBlob.java:1235)[3:org.ops4j.pax.logging.pax-logging-service:1.6.9]
We are using 我们正在使用
Azure-storage-api 1.1.0
Azure存储API 1.1.0
Is this a known bug or limitation? 这是已知的错误或限制吗? What are the scenarios in which we will get this exception?
在什么情况下我们将获得此例外?
We download blobs using following code 我们使用以下代码下载Blob
String storageConnectionString = "DefaultEndpointsProtocol=http;AccountName="+ storageAccount + ";AccountKey=" + primaryAccessKey;
CloudStorageAccount account = CloudStorageAccount.parse(storageConnectionString);
CloudBlobClient blobClient = account.createCloudBlobClient();
CloudBlobContainer container = blobClient.getContainerReference(containerName.toLowerCase());
CloudBlockBlob blockBlob = container.getBlockBlobReference(fileName);
blockBlob.downloadAttributes();
//http://stackoverflow.com/questions/1071858/java-creating-byte-array-whose-size-is-represented-by-a-long
int size = (int)blockBlob.getProperties().getLength();
out = new byte[size];
blockBlob.downloadToByteArray(out, 0);
What is constantly
? 什么是
constantly
? Is it always
, or is it when more than X consumers are trying to fetch the blob? 是
always
,还是当有X个以上的消费者试图获取Blob时?
On the Scalability Targets for Azure Storage you can learn more about the targeted scalability parameters. 在Azure存储的可伸缩性目标上,您可以了解有关目标可伸缩性参数的更多信息。 One of which is target throughput for single blob:
其中之一是单个Blob的目标吞吐量:
Target throughput for single blob Up to 60 MB per second, or up to 500 requests per second
单个Blob的目标吞吐量每秒高达60 MB,或每秒多达500个请求
With your 1000 consumers, there is no doubt you hit that limit when querying the same blob. 对于您的1000个消费者,毫无疑问,当您查询相同的Blob时,您达到了该限制。 Question is - do you really need to get the from the blob so intense, can you cache somewhere (intermediate facede) or can you use CDN (it also works with SAS's )
问题是-您是否真的需要从Blob中获取大量信息,可以将其缓存在某个地方(中间面)还是可以使用CDN(它也可以与SAS一起使用)
If the 1000 consumers are hitting 1000 different blobs, there are are limitations, like: 如果1000个消费者击中1000个不同的Blob,则存在一些限制,例如:
Total Request Rate (assuming 1KB object size) per storage account Up to 20,000 IOPS , entities per second, or messages per second
每个存储帐户的总请求速率 (假设1KB对象大小)高达20,000 IOPS ,每秒实体数或每秒消息数
Which, for the 1000 consumers makes 20 requests per second - based on the number of blocks in your files, it may well also be that limit. 对于1000个使用者来说,每秒发出20个请求-基于文件中的块数,也很可能是该限制。
In any way, you shall revise your application and discover which limit you hit. 无论如何,您都应该修改您的应用程序并发现达到的限制。
This is just to make things clear for someone who reads this question in future, after scanning through all the request urls for download. 这只是为了让以后浏览此问题的人在浏览所有下载请求的URL之后弄清楚。
There were bunch of non-existent blob urls which were causing this exception. 一堆不存在的Blob网址导致了此异常。
声明:本站的技术帖子网页,遵循CC BY-SA 4.0协议,如果您需要转载,请注明本站网址或者原文地址。任何问题请咨询:yoyou2525@163.com.