Azure Storage Blobs - Blob Retention Policies

What are Blob Retention Policies?

Blob retention policies are a crucial feature in Azure Blob Storage that allow you to define how long your data is stored. They help in complying with regulatory requirements, managing storage costs, and preventing accidental data deletion.

Azure Storage offers two main types of retention policies:

Immutable Storage (WORM - Write Once, Read Many): This type of policy enforces a time-based retention period. During this period, blobs are write-protected and cannot be deleted or modified.
Soft Delete: This feature retains deleted blobs for a specified period, allowing you to recover them if they were accidentally deleted.

Types of Retention Policies in Detail

Immutable Storage (Time-Based Retention)

Immutable storage provides a robust way to ensure data integrity and compliance. You can configure a retention period, after which blobs become eligible for deletion.

Key Features:

Configurable retention intervals (days).
Blobs cannot be modified or deleted during the retention period.
Ideal for regulatory compliance (e.g., SEC 17a-4(f), FINRA Rule 4511).
Once a time-based retention policy is set and a blob is committed, it cannot be shortened.

You can set these policies at the container level using Legal Hold or Time-based retention policies.

Soft Delete for Blobs

Soft delete protects your blobs from accidental deletion by keeping them in a recoverable state for a specified duration. This is a safety net for operational data.

Key Features:

Recover deleted blobs within the soft-delete period.
Configurable retention period for deleted blobs (1 to 365 days).
Can be enabled independently or alongside immutable storage.
Soft-deleted blobs incur storage costs until permanently deleted.

Enabling soft delete is recommended for most scenarios to prevent data loss.

Configuring Retention Policies

Retention policies can be configured using the Azure portal, Azure CLI, Azure PowerShell, or Azure SDKs.

Example: Azure CLI for Soft Delete

To enable soft delete for blobs with a 14-day retention period on a storage account:

az storage account update --name <your-storage-account-name> --resource-group <your-resource-group-name> --set isSoftDeleteEnabled=true
az storage blob service-properties update --account-name <your-storage-account-name> --days-retained 14

Example: Azure CLI for Time-Based Retention (Container Level)

To set a 7-day time-based retention policy on a container:

az storage container immutability policy update --container-name <your-container-name> --account-name <your-storage-account-name> --retention-days 7 --policy-mode locked

Note: The --policy-mode locked sets the policy to 'locked' mode, making it immutable and unchangeable.

Carefully consider your retention periods. Once a time-based retention policy is locked, it cannot be shortened or deleted.

Best Practices

Understand your data lifecycle and compliance requirements.
Enable soft delete for all critical data to guard against accidental deletion.
Use immutable storage for data that must be preserved for regulatory or compliance reasons.
Monitor storage costs, especially with soft delete enabled, as retained data still incurs charges.
Regularly test your data recovery process for soft-deleted blobs.

Learn More

For more in-depth information, refer to the official Azure documentation: