What is Azure Event Hubs?

Azure Event Hubs is a big data streaming platform and event ingestion service. It can be very difficult and complex to process streamed data. Event Hubs enables you to process and analyze this stream of data in real time and allows you to store what you want, when you want, and do whatever you want with it, including sending it to an alternative storage or a batch analytics store.

Core Concepts

Event Producers and Consumers

Event Hubs works with two main types of applications:

• Event Producers: Applications that send (produce) events to Event Hubs. These can be IoT devices, web server logs, application telemetry, or any source generating event streams.
• Event Consumers: Applications that read (consume) events from Event Hubs. These can be real-time analytics dashboards, stream processing engines (like Azure Stream Analytics or Apache Spark), or services that archive data.

Partitions

Event Hubs partitions events into a specified number of ordered sequences called partitions. Each partition is a strictly ordered sequence of events. As events arrive, they are added to the end of a specific partition. Event producers can choose which partition to send an event to, or Event Hubs can distribute them automatically. Consumers read from partitions in order.

The number of partitions is a crucial design choice. More partitions generally mean higher throughput, but also require more coordination for consumers. A common strategy is to partition by a specific entity identifier (e.g., a device ID or user ID) to ensure all events for that entity are processed in order.

Consumer Groups

A consumer group represents a unique view of the event stream. Each consumer group allows multiple applications or different instances of the same application to read from Event Hubs independently without interfering with each other. This enables parallel processing and the ability to have different consumption logic for the same data.

Throughput Units (TUs)

Event Hubs capacity is measured in Throughput Units (TUs). A TU provides a defined amount of ingress and egress bandwidth. You can auto-inflate TUs to scale dynamically based on load, or manually set the capacity. Higher tiers of Event Hubs offer more TUs and advanced features.

When to use Azure Event Hubs

Event Hubs is ideal for scenarios involving:

• Real-time analytics: Processing and analyzing streaming data as it arrives.
• Log and telemetry ingestion: Collecting large volumes of logs and telemetry data from distributed sources.
• Data warehousing: Moving large volumes of data into data warehouses for batch processing and analysis.
• Decoupling applications: Acting as a buffer between event producers and consumers, allowing them to operate independently.
• Event sourcing: Storing a log of state-changing events for an application.

Example

Imagine a fleet of IoT devices sending sensor readings. Each device can be configured to send its readings to an Event Hub. A real-time dashboard application can consume these events from a consumer group to display live sensor data, while another application consumes the same events from a different consumer group to store them in a time-series database for historical analysis. Event Hubs ensures that both applications can process the data at their own pace and with their own logic.

// Example C# snippet for sending an event
            using Azure.Messaging.EventHubs;
            using System;
            using System.Text;
            using System.Threading.Tasks;

            // Replace with your Event Hubs connection string and entity path
            string connectionString = "";
            string eventHubName = "";

            await using var producerClient = new EventHubProducerClient(connectionString, eventHubName);

            var eventData = new EventData(Encoding.UTF8.GetBytes("{\"message\": \"Hello Event Hubs!\"}"));

            await producerClient.SendAsync(eventData);
            Console.WriteLine("Event sent successfully.");
            

For more details on sending and receiving data, refer to the Send and receive data quickstart.