Azure Queue Storage

Azure Queue storage is a service that stores large numbers of in-memory messages. You can access queues from anywhere in the world via HTTP or HTTPS. A queue is a collection of messages. Each message can be up to 64 KB in size.

Queue storage is used to store and retrieve messages. This is a common pattern for decoupling applications and for asynchronous processing. For example, when you need to process a large number of items, you can add them to a queue, and then a separate worker process can retrieve and process them at its own pace.

Note: Azure Queue storage is different from Azure Service Bus Queues. Queue storage is designed for simple messaging between application components, while Service Bus Queues are designed for more complex enterprise messaging scenarios.

Key Concepts

  • Queue: A named container for messages.
  • Message: A piece of data (up to 64 KB) stored in a queue.
  • Dequeue: The act of retrieving and removing a message from the queue.
  • Peek: The act of retrieving a message without removing it from the queue.
  • Visibility Timeout: A period during which a dequeued message is invisible to other clients attempting to dequeue.

Common Scenarios

  • Decoupling application components: Allows different parts of an application to communicate asynchronously.
  • Asynchronous task processing: Offload long-running tasks to background workers.
  • Batch processing: Handle large volumes of data by queuing them for sequential processing.
  • Load balancing: Distribute incoming requests across multiple worker instances.

Core Operations

The primary operations for Azure Queue Storage include:

  • Creating a queue.
  • Adding a message to a queue.
  • Getting messages from a queue (and optionally making them invisible).
  • Peeking at messages.
  • Deleting messages.
  • Clearing a queue.
  • Deleting a queue.

API Reference (Conceptual)

Here's a conceptual overview of some common REST API operations:

Operation HTTP Method Path Description
Create Queue PUT /<queue-name> Creates a new queue.
Enqueue Message POST /<queue-name>/messages Adds a new message to the queue.
Dequeue Messages GET /<queue-name>/messages Retrieves one or more messages from the queue.
Peek Messages GET /<queue-name>/messages?peekonly=true Retrieves one or more messages without making them invisible.
Delete Message DELETE /<queue-name>/messages/<message-id> Deletes a specific message.
Clear Queue DELETE /<queue-name>/messages Deletes all messages from the queue.
Delete Queue DELETE /<queue-name> Deletes the queue.

Using SDKs

Azure provides SDKs for various languages to interact with Queue Storage more easily. Here's a snippet using the Azure SDK for Python:


from azure.storage.queue import QueueServiceClient

# Replace with your actual connection string
connection_string = "YOUR_AZURE_STORAGE_CONNECTION_STRING"
queue_name = "my-message-queue"

# Create a QueueServiceClient
queue_service_client = QueueServiceClient.from_connection_string(connection_string)

# Get a client to interact with the queue
queue_client = queue_service_client.get_queue_client(queue_name)

# Create the queue if it doesn't exist
try:
    queue_client.create_queue()
    print(f"Queue '{queue_name}' created.")
except Exception as e:
    print(f"Queue '{queue_name}' might already exist: {e}")

# Enqueue a message
message_text = "Hello, Azure Queue Storage!"
queue_client.send_message(message_text)
print(f"Message '{message_text}' sent to queue.")

# Dequeue a message
messages = queue_client.receive_messages()
if messages:
    for msg in messages:
        print(f"Received message: {msg.content}")
        # Delete the message after processing
        queue_client.delete_message(msg.id, msg.pop_receipt)
        print("Message deleted.")
else:
    print("No messages in the queue.")
Tip: For production scenarios, always use generated SAS tokens or managed identities for secure access to your storage accounts instead of hardcoding connection strings.

Limitations

  • Maximum message size is 64 KB.
  • Queues and messages have limits on the number of items and size.
  • Queue Storage is not suitable for scenarios requiring complex transaction support or guaranteed ordering across multiple consumers without additional application logic.

Further Reading