Events in Azure Event Hubs

Events are the fundamental units of data processed by Azure Event Hubs. They represent a discrete, immutable sequence of bytes that are captured and made available for consumption. Understanding the structure and properties of events is crucial for effectively building event-driven applications with Event Hubs.

Conceptual flow of events through Event Hubs.

Event Structure

An event, often referred to as an event payload or simply message, consists of the following key components:

• Body: The actual data payload of the event. This can be any binary data, such as JSON, Avro, Protobuf, or plain text. The body is typically what your application logic will process.
• Properties: A collection of key-value pairs that provide metadata about the event. These properties can be used for routing, filtering, or adding contextual information. Event Hubs supports system properties and custom application properties.

System Properties

Event Hubs assigns several system properties to events when they are sent. These include:

• MessageId: A unique identifier for the event within the partition.
• SequenceNumber: A sequential number assigned to the event within the partition.
• Offset: The offset of the event within the partition's log. This is a unique identifier for the event.
• PartitionKey: Used to ensure event ordering within a specific partition. If provided, events with the same PartitionKey will always be sent to the same partition.
• EnqueuedTimeUtc: The UTC timestamp when the event was enqueued into Event Hubs.
• Properties: A dictionary containing custom application properties.

Custom Properties

You can attach custom metadata to your events using key-value pairs. This is invaluable for adding business context, routing hints, or identifiers that your consumers can use to process events.

Event Serialization

While Event Hubs treats the event body as raw bytes, the way you serialize and deserialize this data is application-specific. Common serialization formats include:

• JSON: Widely used for its human-readability and broad language support.
• Avro: A compact binary format that supports schema evolution, making it ideal for large-scale data processing.
• Protobuf: Another efficient binary format, often used for performance-critical applications.

Event Ordering

Event Hubs guarantees ordering of events within a partition. If you need strict ordering across all events, you should design your application to send all related events to the same partition, often by using a consistent PartitionKey.

Example: Sending an Event

Here's a conceptual example of how an event might be constructed and sent (using a hypothetical SDK syntax):

// C# Example (conceptual)
var eventData = new EventData(Encoding.UTF8.GetBytes("{\"sensorId\": \"abc-123\", \"temperature\": 25.5}"));
eventData.Properties.Add("contentType", "application/json");
eventData.Properties.Add("sourceSystem", "IoTDevice");
eventData.PartitionKey = "device-group-1"; // Ensures this event goes to the same partition as other events from this group

await eventHubProducerClient.SendEventAsync(eventData);
        

Key Takeaways

• Events are the core data units in Event Hubs.
• Each event contains a body and optional properties.
• PartitionKey is essential for guaranteeing event ordering within a partition.
• Choose an appropriate serialization format for your event body.
• Event Hubs provides system properties, and you can add custom properties for metadata.