Copying one Azure blob to another blob in Azure Storage Client 2.0

Question

In the old 1.7 storage client there was a CloudBlob.CopyFromBlob(otherBlob) method, but it does not seem to be present in the 2.0 version. What is the recommended best practice for copying blobs? I do see a ICloudBlob.BeginStartCopyFromBlob method. If that is the appropriate method, how do I use it?

Answer 1

Gaurav Mantri has written a series of articles on Azure Storage on version 2.0. I have taken this code extract from his blog post of Storage Client Library 2.0 – Migrating Blob Storage Code for Blob Copy

CloudStorageAccount storageAccount = new CloudStorageAccount(new StorageCredentials(accountName, accountKey), true);
CloudBlobClient cloudBlobClient = storageAccount.CreateCloudBlobClient();
CloudBlobContainer sourceContainer = cloudBlobClient.GetContainerReference(containerName);
CloudBlobContainer targetContainer = cloudBlobClient.GetContainerReference(targetContainerName);
string blobName = "<Blob Name e.g. myblob.txt>";
CloudBlockBlob sourceBlob = sourceContainer.GetBlockBlobReference(blobName);
CloudBlockBlob targetBlob = targetContainer.GetBlockBlobReference(blobName);
targetBlob.StartCopyFromBlob(sourceBlob);

Answer 2

Using Storage 6.3 (much newer library than in original question) and async methods use StartCopyAsync ( MSDN )

  CloudStorageAccount storageAccount = CloudStorageAccount.Parse("Your Connection");

  CloudBlobClient blobClient = storageAccount.CreateCloudBlobClient();
  CloudBlobContainer container = blobClient.GetContainerReference("YourContainer");

  CloudBlockBlob source = container.GetBlockBlobReference("Your Blob");
  CloudBlockBlob target = container.GetBlockBlobReference("Your New Blob");
        
  await target.StartCopyAsync(source);

Answer 3

FYI as of the latest version (7.x) of the SDK this no longer works because the BeginStartCopyBlob function no longer exists.

You can do it this way:

// this tunnels the data via your program,
// so it reuploads the blob instead of copying it on service side
using (var stream = await sourceBlob.OpenReadAsync())
{
  await destinationBlob.UploadFromStreamAsync(stream);
}

As mentioned by @(Alexey Shcherbak) this is a better way to proceed:

await targetCloudBlob.StartCopyAsync(sourceCloudBlob.Uri);

while (targetCloudBlob.CopyState.Status == CopyStatus.Pending)
{
    await Task.Delay(500);
    // Need to fetch or "CopyState" will never update
    await targetCloudBlob.FetchAttributesAsync(); 
}

if (targetCloudBlob.CopyState.Status != CopyStatus.Success)
{
    throw new Exception("Copy failed: " + targetCloudBlob.CopyState.Status);
}

Answer 4

Starting Azure Storage 8, to move Blobs between Storage Accounts I use code similar to below, hope it will help somebody:

//copy blobs - from
CloudStorageAccount sourceStorageAccount = new CloudStorageAccount(new StorageCredentials(storageFromName, storageFromKey), true);
CloudBlobClient sourceCloudBlobClient = sourceStorageAccount.CreateCloudBlobClient();
CloudBlobContainer sourceContainer = sourceCloudBlobClient.GetContainerReference(containerFromName);

//copy blobs - to
CloudStorageAccount targetStorageAccount = new CloudStorageAccount(new StorageCredentials(storageToName, storageToKey), true);
CloudBlobClient targetCloudBlobClient = targetStorageAccount.CreateCloudBlobClient();
CloudBlobContainer targetContainer = targetCloudBlobClient.GetContainerReference(containerToName);

//create target container if didn't exists
try{
    await targetContainer.CreateIfNotExistsAsync();
}
catch(Exception e){
    log.Error(e.Message);
}

CloudBlockBlob sourceBlob = sourceContainer.GetBlockBlobReference(blobName);
CloudBlockBlob targetBlob = targetContainer.GetBlockBlobReference(blobName);

try{
    //initialize copying
    await targetBlob.StartCopyAsync(sourceBlob.Uri);
}
catch(Exception ex){
    log.Error(ex.Message);
    //return error, in my case HTTP
    return req.CreateResponse(HttpStatusCode.BadRequest, "Error, source BLOB probably has private access only: " +ex.Message);
} 

//fetch current attributes
targetBlob.FetchAttributes();

//waiting for completion
while (targetBlob.CopyState.Status == CopyStatus.Pending){
    log.Info("Status: " + targetBlob.CopyState.Status);
    Thread.Sleep(500);
    targetBlob.FetchAttributes();
}

//check status
if (targetBlob.CopyState.Status != CopyStatus.Success){
    //return error, in my case HTTP
    return req.CreateResponse(HttpStatusCode.BadRequest, "Copy failed with status: " + targetBlob.CopyState.Status);
}

//finally remove source in case Copy Status was Success
sourceBlob.Delete();

//and return success (in my case HTTP)
return req.CreateResponse(HttpStatusCode.OK, "Done.");

Answer 5

Naveen already explained the correct syntax for using StartCopyFromBlob (the synchronous method). The method you mentioned ( BeginStartCopyFromBlob ) is the asynchronous alternative which you can use in combination with a Task for example:

    var blobClient = account.CreateCloudBlobClient();

    // Upload picture.
    var picturesContainer = blobClient.GetContainerReference("pictures");
    picturesContainer.CreateIfNotExists();
    var myPictureBlob = picturesContainer.GetBlockBlobReference("me.png");
    using (var fs = new FileStream(@"C:\Users\Public\Pictures\Sample Pictures\Chrysanthemum.jpg", FileMode.Open))
        myPictureBlob.UploadFromStream(fs);

    // Backup picture.
    var backupContainer = blobClient.GetContainerReference("backup");
    backupContainer.CreateIfNotExists();
    var backupBlob = picturesContainer.GetBlockBlobReference("me.png");

    var task = Task.Factory.FromAsync<string>(backupBlob.BeginStartCopyFromBlob(myPictureBlob, null, null), backupBlob.EndStartCopyFromBlob);
    task.ContinueWith((t) =>
    {
        if (!t.IsFaulted)
        {
            while (true)
            {
                Console.WriteLine("Copy state for {0}: {1}", backupBlob.Uri, backupBlob.CopyState.Status);
                Thread.Sleep(500);
            }
        }
        else
        {
            Console.WriteLine("Error: " + t.Exception);
        }
    });

Answer 6

For me, WindowsAzure.Storage 8.0.1, James Hancock's solution did the server side copy but the client copy status was stuck on Pending (looping forever). Solution was to call FetchAttributes() on targetCloudBlob after Thread.sleep(500) .

// Aaron Sherman's code 

targetCloudBlob.StartCopy(sourceCloudBlob.Uri);

while (targetCloudBlob.CopyState.Status == CopyStatus.Pending)
{
    Thread.Sleep(500);
    targetCloudBlob.FetchAttributes();
}

// James Hancock's remaining code

Official Microsoft documentation (async example)

Answer 7

It seems that the API might have been cleaned up a little since previous posts were made.

// _client is a BlobServiceClient injected via DI in the constructor.

BlobContainerClient sourceContainerClient = _client.GetBlobContainerClient(sourceContainerName);
BlobClient sourceClient = sourceContainerClient.GetBlobClient(blobName);

BlobContainerClient destContainerClient = _client.GetBlobContainerClient(destContainerName);
BlobClient destClient = destContainerClient.GetBlobClient(blobName);

// assume that if the following doesn't throw an exception, then it is successful.
CopyFromUriOperation operation = await destClient.StartCopyFromUriAsync(sourceClient.Uri);
await operation.WaitForCompletionAsync();

The documentation for operation.WaitForCompletionAsync says:

Periodically calls the server till the long-running operation completes. This method will periodically call UpdateStatusAsync till HasCompleted is true, then return the final result of the operation.

Reviewing the source code for this method seems to call BlobBaseClient.GetProperties (or the async version) which will throw an RequestFailureException on error.

Answer 8

here is my short simple answer.

public void Copy(CloudBlockBlob srcBlob, CloudBlobContainer destContainer)
{
    CloudBlockBlob destBlob;

    if (srcBlob == null)
    {
        throw new Exception("Source blob cannot be null.");
    }

    if (!destContainer.Exists())
    {
        throw new Exception("Destination container does not exist.");
    }

    //Copy source blob to destination container
    string name = srcBlob.Uri.Segments.Last();
    destBlob = destContainer.GetBlockBlobReference(name);
    destBlob.StartCopyAsync(srcBlob);                
}