Azure Blob storage batch update. Possible?

Question

I need to update blobs metadata in an existing container. I currently have more than 5k blobs in one container. What I do as of now:

• blobs are received via container.ListBlobsSegmentedAsync() with BlobContinuationToken

• then I'm aggregating the tasks from blob.UpdateMetadataAsync() and run them concurrently via Task.WhenAll()

All in all, it takes around 10 seconds on my machine for the whole process. But as I understand, I'm creating a pressure on the network as I need to send 5k requests to the Azure and it needs to process all of them.

Is there a better way? Can I send a batch request to do the task or I'm left with the current approach?

Thanks!

Answer 1

Can I send a batch request to do the task or I'm left with the current approach?

I don't think that there is a way to send a batch request to do that.

From the Azure storage blob Rest API , you could know that there is no such Rest API.

But if you have any ideas about Azure storage, you could give your feedback to Azure team.

Sure. you could use the async and concurrently way to update the blobs.

Is there a better way?

If you don't want to send the such much requests from client.

I recommand that you could do that with Azure function http trigger . And implement the updating blobs logic in the Azure function.

For more information about Azure function, please refer to this document .

Answer 2

The Batch API for Blob Storage is now in preview . Unfortunately for now, it only support DeleteBlob and SetBlobTier operations but support for more operations is coming.

Documentation: https://docs.microsoft.com/en-us/rest/api/storageservices/blob-batch