Copying Blobs in Azure Storage

This document provides a comprehensive guide on how to copy blobs within Azure Storage. Copying blobs is a fundamental operation for data migration, backup, and replication within your Azure Storage account or across different accounts.

Prerequisites

  • An Azure account with an active subscription.
  • An Azure Storage account (or accounts).
  • Azure CLI, Azure PowerShell, or an Azure SDK installed and configured.
  • Appropriate permissions to access the source and destination storage accounts and blobs.

Copy Blob Scenarios

Azure Blob Storage supports several scenarios for copying blobs:

  • Copying a blob from one storage account to another.
  • Copying a blob from one container to another within the same storage account.
  • Copying a blob from a URL (e.g., a public URL or a SAS URL).

Copying a Blob Between Storage Accounts

This is a common requirement for disaster recovery, data archiving, or moving data to a different region.

Using Azure CLI

The az storage blob copy start command is used to initiate a blob copy operation.


az storage blob copy start \
    --account-name <source_account_name> \
    --account-key <source_account_key> \
    --container-name <source_container_name> \
    --name <source_blob_name> \
    --destination-account-name <destination_account_name> \
    --destination-container <destination_container_name> \
    --destination-blob <destination_blob_name>

Replace the placeholders with your actual storage account names, keys, container names, and blob names.

Using PowerShell

The Start-AzStorageBlobCopy cmdlet facilitates copying blobs between storage accounts.


$sourceContext = New-AzStorageContext -StorageAccountName "<source_account_name>" -StorageAccountKey "<source_account_key>"
$destinationContext = New-AzStorageContext -StorageAccountName "<destination_account_name>" -StorageAccountKey "<destination_account_key>"

Start-AzStorageBlobCopy `
    -SrcContainer "<source_container_name>" `
    -SrcBlob "<source_blob_name>" `
    -DestContainer "<destination_container_name>" `
    -DestBlob "<destination_blob_name>" `
    -Context $sourceContext `
    -DestContext $destinationContext

Using .NET SDK

The Azure Storage Blob client library for .NET makes this operation straightforward.


using Azure.Storage.Blobs;
using Azure.Storage.Blobs.Models;

string sourceConnectionString = "DefaultEndpointsProtocol=https;AccountName=<source_account_name>;AccountKey=<source_account_key>;EndpointSuffix=core.windows.net";
string destinationConnectionString = "DefaultEndpointsProtocol=https;AccountName=<destination_account_name>;AccountKey=<destination_account_key>;EndpointSuffix=core.windows.net";

BlobServiceClient sourceServiceClient = new BlobServiceClient(sourceConnectionString);
BlobServiceClient destinationServiceClient = new BlobServiceClient(destinationConnectionString);

string sourceContainerName = "<source_container_name>";
string sourceBlobName = "<source_blob_name>";
string destinationContainerName = "<destination_container_name>";
string destinationBlobName = "<destination_blob_name>";

BlobClient sourceBlobClient = sourceServiceClient.GetBlobContainerClient(sourceContainerName).GetBlobClient(sourceBlobName);
BlobContainerClient destinationContainerClient = destinationServiceClient.GetBlobContainerClient(destinationContainerName);

// Ensure the destination container exists
destinationContainerClient.CreateIfNotExists();

BlobClient destinationBlobClient = destinationContainerClient.GetBlobClient(destinationBlobName);

// Start the copy operation
CopyFromUriOperation operation = await destinationBlobClient.StartCopyFromUriAsync(sourceBlobClient.Uri);

// Optionally, wait for the copy to complete
await operation.UpdateStatusAsync();
if (operation.HasCompleted)
{
    Console.WriteLine($"Blob copied successfully. Status: {operation.Status}");
}
else
{
    Console.WriteLine($"Blob copy operation is still in progress. Status: {operation.Status}");
}

Using REST API

The Blob Copy REST API provides granular control over the operation.

Request URL:


PUT /{destination_container_name}/{destination_blob_name}?comp=copy HTTP/1.1
Host: {destination_account_name}.blob.core.windows.net
x-ms-version: 2020-08-04
Content-Length: 0
x-ms-copy-source: /{source_account_name}/{source_container_name}/{source_blob_name}
Authorization: SharedKey {destination_account_name}:{signature}
Date: {date}

You'll need to construct the appropriate authorization header.

Copying a Blob Within a Storage Account

This scenario is useful for creating backups or different versions of a blob within the same account.

Using Azure CLI

The process is similar to copying between accounts, but you only need to specify one account context.


az storage blob copy start \
    --account-name <storage_account_name> \
    --account-key <storage_account_key> \
    --container-name <source_container_name> \
    --name <source_blob_name> \
    --destination-container <destination_container_name> \
    --destination-blob <destination_blob_name>

Using PowerShell


$ctx = New-AzStorageContext -StorageAccountName "<storage_account_name>" -StorageAccountKey "<storage_account_key>"

Start-AzStorageBlobCopy `
    -SrcContainer "<source_container_name>" `
    -SrcBlob "<source_blob_name>" `
    -DestContainer "<destination_container_name>" `
    -DestBlob "<destination_blob_name>" `
    -Context $ctx

Copying a Blob from a URL

You can copy a blob directly from a URL, such as a public URL or a URL with a Shared Access Signature (SAS).

This is particularly useful for migrating data from other cloud providers or web sources into Azure Blob Storage.

Azure CLI Example:


az storage blob copy start \
    --destination-container <destination_container_name> \
    --destination-blob <destination_blob_name> \
    --account-name <destination_account_name> \
    --account-key <destination_account_key> \
    --copy-source <source_url_with_sas_if_needed>

.NET SDK Example:


// ... (setup destinationBlobClient as before)

string sourceUri = "<public_or_sas_url_of_source_blob>";
CopyFromUriOperation operation = await destinationBlobClient.StartCopyFromUriAsync(new Uri(sourceUri));
// ... (wait for completion)

Copying Blobs with Metadata

By default, a blob copy operation does not copy the source blob's metadata. You can choose to copy the metadata or set new metadata for the destination blob.

When using SDKs or the REST API, you can specify metadata during the copy operation or by updating the blob properties after the copy.

Azure CLI: Metadata can be set via --metadata parameter during copy, or by using az storage blob metadata update after the copy.

.NET SDK: Pass a BlobCopyFromUriOptions object to StartCopyFromUriAsync to specify metadata.

Controlling Overwrites

If a blob with the same name already exists in the destination container, the behavior depends on the method used and its parameters. Most methods allow you to control whether the existing blob should be overwritten.

Caution: Be careful when copying to a destination that might contain existing blobs. Always verify your destination blob names to avoid unintentional data loss.

Concurrency and Asynchronous Operations

Blob copy operations are asynchronous. When you initiate a copy, the Azure Storage service returns a copy ID. You can poll the status of the copy operation using this ID or the blob's properties.

The x-ms-copy-status header in the response indicates the current state: pending, success, or aborted.

Conclusion

Copying blobs in Azure Storage is a versatile operation that can be performed using various tools and programming languages. Understanding these methods allows for efficient data management, migration, and backup strategies for your cloud-based data.