Using Azure Storage Queues with Python

This document provides a comprehensive guide on how to interact with Azure Storage Queues using the Python SDK. Azure Queues are a service that enables you to store large numbers of messages that can be accessed from anywhere in the world via HTTP or HTTPS. A single queue message can be up to 64 KB in size.

Azure Queues are a component of Azure Storage. You'll need an Azure Storage account to use queues.

1. Installation

First, you need to install the Azure Storage Blob client library for Python. You can do this using pip:

                bash
                pip install azure-storage-queue
            

2. Authentication

To interact with your Azure Storage Queue, you'll need a connection string or a shared access signature (SAS) token. The easiest way is often to use the connection string from your storage account.

2.1. Using Connection String

You can retrieve your storage account connection string from the Azure portal under your storage account's "Access keys" section.

                python
                
from azure.storage.queue import QueueClient

connect_str = "YOUR_AZURE_STORAGE_CONNECTION_STRING"
queue_name = "my-sample-queue"

queue_client = QueueClient.from_connection_string(connect_str, queue_name)

2.2. Using SAS Token

Alternatively, you can generate a SAS token with appropriate permissions for more granular control.

                python
                
from azure.storage.queue import QueueClient, AccountSasPermissions, ResourceTypes, AccountKind

# Assume you have your account name and key, and generate a SAS token
# For simplicity, we'll use connection string here, but SAS is also an option.
# Refer to Azure SDK documentation for detailed SAS token generation.

3. Creating a Queue

You can create a new queue if it doesn't already exist.

                python
                
try:
    queue_client.create_queue()
    print(f"Queue '{queue_name}' created successfully.")
except Exception as e:
    print(f"Error creating queue: {e}")
                
            

4. Sending Messages

To add a message to the queue, use the send_message method. Messages can be strings or bytes.

                python
                
message_text = "Hello, Azure Queues!"
response = queue_client.send_message(message_text)
print(f"Message sent with ID: {response['id']}")

# Send multiple messages
messages_to_send = ["Task 1: Process data", "Task 2: Send notification", "Task 3: Log activity"]
responses = queue_client.send_message(messages_to_send)
for resp in responses:
    print(f"Message sent with ID: {resp['id']}")
                
            

5. Receiving Messages

You can retrieve messages from the queue using receive_message. This makes the message invisible to other clients for a specified period (visibility timeout).

                python
                
# Receive a single message
message = queue_client.receive_message()
if message:
    print(f"Received message: {message['content']}")
    print(f"Message ID: {message['id']}")
    print(f"Pop Receipt: {message['pop_receipt']}")

# Receive multiple messages
messages = queue_client.receive_message(messages_per_page=5)
for msg in messages:
    print(f"Received message: {msg['content']}")
    print(f"Message ID: {msg['id']}")
    print(f"Pop Receipt: {msg['pop_receipt']}")
                
            

6. Deleting Messages

Once a message has been processed, it must be explicitly deleted from the queue. You need the message's pop_receipt and message_id.

                python
                
# Assuming 'message' is the object from receive_message()
if message:
    queue_client.delete_message(message['id'], message['pop_receipt'])
    print(f"Message with ID {message['id']} deleted.")
                
            

7. Peeking Messages

You can peek at messages without making them invisible. This is useful for inspecting messages without consuming them.

                python
                
peeked_messages = queue_client.peek_messages(messages_per_page=3)
for msg in peeked_messages:
    print(f"Peeked message content: {msg['content']}")

8. Clearing a Queue

To remove all messages from a queue, use the clear_messages method.

                python
                
queue_client.clear_messages()
print(f"All messages cleared from queue '{queue_name}'.")

9. Deleting a Queue

To remove a queue and all its messages permanently, use delete_queue.

                python
                
# Make sure you want to delete the queue permanently!
# queue_client.delete_queue()
# print(f"Queue '{queue_name}' deleted.")

10. Advanced Operations

The Python SDK also supports setting queue metadata, retrieving queue properties, and managing message time-to-live (TTL) and visibility timeouts.

Queue Properties: Get the approximate number of messages and queue metadata.
Message TTL: Control how long messages persist in the queue.
Visibility Timeout: Adjust the time a message remains invisible after being dequeued.

Refer to the official Azure Queues documentation and the Python SDK repository for more details and advanced usage patterns.

Consider implementing retry logic for message processing to handle transient failures gracefully.