Configure Azure Blob Storage Lifecycle Management with Azure CLI

Configure Storage Lifecycle Management Rules with Azure CLI

Azure Blob Storage lifecycle management allows you to manage your data throughout its lifecycle. You can transition data to cooler storage tiers (like cool or archive) or delete it when it's no longer needed. This section focuses on configuring these rules using the Azure Command-Line Interface (CLI).

Using Azure CLI

The Azure CLI provides powerful commands to manage your Azure resources, including setting up lifecycle management policies. You'll typically use the az storage lifecycle-management policy create and az storage lifecycle-management policy update commands.

Step 1: Create a JSON file for your policy

Define your lifecycle rules in a JSON file. This file specifies the actions to take based on rules like object age or creation date.

{
    "rules": [
        {
            "enabled": true,
            "name": "move_to_cool",
            "type": "Lifecycle",
            "definition": {
                "actions": {
                    "baseBlob": {
                        "tierToCool": {
                            "daysAfterModificationGreaterThan": 30
                        }
                    },
                    "snapshot": {
                        "tierToCool": {
                            "daysAfterCreationGreaterThan": 30
                        }
                    }
                },
                "filters": {
                    "blobTypes": [
                        "blockBlob"
                    ],
                    "prefixMatch": [
                        "logs/"
                    ]
                }
            }
        },
        {
            "enabled": true,
            "name": "delete_old_logs",
            "type": "Lifecycle",
            "definition": {
                "actions": {
                    "baseBlob": {
                        "delete": {
                            "daysAfterModificationGreaterThan": 365
                        }
                    }
                },
                "filters": {
                    "blobTypes": [
                        "blockBlob"
                    ],
                    "prefixMatch": [
                        "logs/"
                    ]
                }
            }
        }
    ]
}

Step 2: Apply the policy to your storage account

Use the az storage lifecycle-management policy create command to apply the policy defined in your JSON file to a specific storage account.

az storage lifecycle-management policy create \
    --account-name  \
    --resource-group  \
    --policy @policy.json

Replace <your-storage-account-name> and <your-resource-group-name> with your actual resource and storage account names. The @policy.json part tells the CLI to read the policy from the specified file.

Step 3: View or Update the policy

You can view the current policy with:

az storage lifecycle-management policy show \
    --account-name  \
    --resource-group

To update an existing policy, you can use the update command with a new JSON file or by specifying changes directly (though using a file is often cleaner for complex policies).

az storage lifecycle-management policy update \
    --account-name  \
    --resource-group  \
    --policy @new-policy.json

Note: Lifecycle management rules are evaluated once a day. It may take up to 24 hours for changes to take effect.

Using Azure PowerShell

Azure PowerShell also offers cmdlets for managing lifecycle policies. You'll use New-AzStorageLifecyclePolicy or Set-AzStorageLifecyclePolicy.

Step 1: Create a PowerShell object or JSON file for your policy

Similar to the CLI, you can define your policy structure.

Example using a JSON file:

# Assuming policy.json contains the same JSON as the CLI example
                    Set-AzStorageLifecyclePolicy -ResourceGroupName "" -StorageAccountName "" -Policy "policy.json"

Alternatively, you can construct the policy object directly in PowerShell:

$policyObject = New-Object -TypeName Microsoft.Azure.Commands.Management.Storage.Models.PSStorageAccountLifecyclePolicy
                    $rule = New-Object -TypeName Microsoft.Azure.Commands.Management.Storage.Models.PSStorageAccountLifecyclePolicyRule
                    $rule.Name = "move_to_cool"
                    $rule.Enabled = $true
                    $rule.Type = "Lifecycle"
                    $rule.Definition = New-Object -TypeName Microsoft.Azure.Commands.Management.Storage.Models.PSStorageAccountLifecyclePolicyDefinition
                    $rule.Definition.Actions = New-Object -TypeName Microsoft.Azure.Commands.Management.Storage.Models.PSStorageAccountLifecyclePolicyActions
                    $rule.Definition.Actions.BaseBlob = New-Object -TypeName Microsoft.Azure.Commands.Management.Storage.Models.PSStorageAccountLifecyclePolicyBaseBlobActions
                    $rule.Definition.Actions.BaseBlob.TierToCool = New-Object -TypeName Microsoft.Azure.Commands.Management.Storage.Models.PSStorageAccountLifecyclePolicyTierToCool
                    $rule.Definition.Actions.BaseBlob.TierToCool.DaysAfterModificationGreaterThan = 30
                    # ... configure other rules and filters similarly ...
                    $policyObject.Rules.Add($rule)

                    Set-AzStorageLifecyclePolicy -ResourceGroupName "" -StorageAccountName "" -Policy $policyObject

Step 2: Apply the policy

Set-AzStorageLifecyclePolicy `
    -ResourceGroupName "" `
    -StorageAccountName "" `
    -Policy "policy.json"

Step 3: View or Update

To view:

Get-AzStorageLifecyclePolicy -ResourceGroupName "" -StorageAccountName ""

Use Set-AzStorageLifecyclePolicy again to update.

Using the REST API

You can also manage lifecycle policies programmatically using the Azure Storage Lifecycle Management REST API. This involves making HTTP requests to the Azure Storage management endpoint.

Endpoint

The endpoint for managing lifecycle policies is:

PUT https://management.azure.com/subscriptions/{subscriptionId}/resourceGroups/{resourceGroupName}/providers/Microsoft.Storage/storageAccounts/{accountName}/lifecyclePolicies/default?api-version=2021-04-01

Request Body (JSON)

The request body is the same JSON structure used for the Azure CLI and PowerShell.

{
    "properties": {
        "rules": [
            {
                "enabled": true,
                "name": "move_to_cool",
                "type": "Lifecycle",
                "definition": {
                    "actions": {
                        "baseBlob": {
                            "tierToCool": {
                                "daysAfterModificationGreaterThan": 30
                            }
                        },
                        "snapshot": {
                            "tierToCool": {
                                "daysAfterCreationGreaterThan": 30
                            }
                        }
                    },
                    "filters": {
                        "blobTypes": [
                            "blockBlob"
                        ],
                        "prefixMatch": [
                            "logs/"
                        ]
                    }
                }
            }
        ]
    }
}

Example using cURL

curl -X PUT \
  'https://management.azure.com/subscriptions//resourceGroups//providers/Microsoft.Storage/storageAccounts//lifecyclePolicies/default?api-version=2021-04-01' \
  -H 'Content-Type: application/json' \
  -H 'Authorization: Bearer ' \
  -d '{
    "properties": {
        "rules": [
            {
                "enabled": true,
                "name": "move_to_cool",
                "type": "Lifecycle",
                "definition": {
                    "actions": {
                        "baseBlob": {
                            "tierToCool": {
                                "daysAfterModificationGreaterThan": 30
                            }
                        }
                    },
                    "filters": {
                        "blobTypes": [
                            "blockBlob"
                        ]
                    }
                }
            }
        ]
    }
}'

You'll need to obtain an access token for Azure Resource Manager to authenticate the request.

Key Concepts and Best Practices

Rules: Each rule consists of actions and filters.
Actions: Define what happens to blobs (e.g., tier to cool, tier to archive, delete).
Filters: Specify which blobs the rule applies to (e.g., based on blob type, prefix, or tags).
Tiering: Transitioning blobs to cooler tiers reduces storage costs but incurs higher access costs.
Deletion: Permanently removes blobs. Ensure this aligns with your data retention policies.
Versioning: Lifecycle management can also be applied to blob versions.
Immutability: Lifecycle rules can be configured to work with immutable storage policies.

Tip: Start with simple rules and gradually add complexity. Test your rules thoroughly in a non-production environment before applying them to critical data.