Azure Storage Queues: Concepts
This document explains the core concepts of Azure Storage Queues.
What is Azure Queue Storage?
Azure Queue storage is a service that stores large numbers of messages that can be accessed from anywhere in the world via authenticated calls using HTTP or HTTPS. Each message in a queue is approximately 64 KB in size, and a queue can contain an unlimited number of messages. Queue storage is commonly used to:
- Decouple application components that run independently.
- Scale work across multiple instances of an application.
- Process messages asynchronously.
Core Components of Queue Storage
Queues
A queue is a collection of messages. Each message in a queue is stored in Base64 encoded format. Queues are typically used to store messages for asynchronous processing. Azure Queue storage supports two types of queues:
- Public Queues: Accessible by anyone on the internet.
- Private Queues: Accessible only by authenticated users or applications with appropriate permissions.
Messages
A message is a unit of data that is stored in a queue. The maximum size of a message is 64 KB. Messages are processed in a First-In, First-Out (FIFO) manner. When a message is added to a queue, it becomes available for consumption. Once a consumer retrieves a message, it becomes invisible to other consumers for a specified period (visibility timeout).
Message States
Messages in Azure Queue storage can be in one of the following states:
- Available: The message is in the queue and can be dequeued.
- Invisible: The message has been dequeued but has not yet been deleted. It will become available again after its visibility timeout expires.
- Deleted: The message has been successfully processed and deleted from the queue.
Account Name and Account Key
To interact with Azure Storage, you need an Azure Storage account. Each storage account is uniquely identified by a name. Access to a storage account is controlled by account keys, which provide authorization for operations. It's crucial to protect your account keys.
SAS (Shared Access Signature)
A Shared Access Signature (SAS) provides a delegated, secure access to your Azure Storage resources without needing to share your account access keys. You can grant clients permissions to specific queues or messages for a limited time, with specific HTTP methods, and under specific IP address restrictions.
Queue Operations
Key operations for Queue Storage include:
- Enqueuing a message: Adding a message to the queue.
- Dequeuing a message: Retrieving and making a message invisible for processing.
- Peeking at messages: Viewing messages without making them invisible.
- Updating a message: Extending the visibility timeout of a dequeued message.
- Deleting a message: Permanently removing a processed message.
- Getting queue metadata: Retrieving the approximate number of messages in a queue.
Use Cases
Queue Storage is well-suited for scenarios such as:
- Task Queues: Distributing work items to multiple worker instances.
- Message Buffering: Handling bursts of traffic or temporary unavailability of downstream services.
- Asynchronous Processing: Offloading long-running operations from user request threads.
Comparison with Other Azure Messaging Services
While Azure Queue Storage is excellent for simple message queuing, for more advanced messaging patterns like pub/sub, ordered delivery, or complex transaction handling, consider using Azure Service Bus or Azure Event Hubs.
Visibility Timeout
When a message is dequeued, it is not immediately deleted. Instead, it becomes invisible to other consumers for a configurable period, known as the visibility timeout. If the consumer successfully processes the message within this timeout, it can explicitly delete the message. If the timeout expires before the message is deleted, the message becomes visible again and can be dequeued by another consumer.
Example: A web application enqueues a task. A worker instance dequeues the task, processes it, and then deletes it.
If the worker fails before deleting, the task becomes available again after the visibility timeout.