Azure Event Hubs: Basic Producer-Consumer (C#)

Table of Contents

Introduction

This sample demonstrates a basic producer and consumer application using Azure Event Hubs with C#. The producer sends events to an Event Hub, and the consumer reads and processes those events.

This is a fundamental pattern for building real-time data processing pipelines, microservices communication, and telemetry ingestion.

Prerequisites

Setup

1. Create a new .NET Console Application:

Command Line

dotnet new console -n EventHubsSample
cd EventHubsSample
Copied!

2. Add the Azure Event Hubs SDK NuGet package:

Command Line

dotnet add package Azure.Messaging.EventHubs
Copied!

3. Configure your connection string:

It's recommended to use environment variables or a configuration file for sensitive information like connection strings. For this sample, we'll assume you'll set an environment variable named EVENTHUB_CONNECTION_STRING.

Security Note: Never hardcode connection strings in your application code, especially in production.

Producer Code

Replace the contents of Program.cs with the following producer code:

using Azure.Messaging.EventHubs;
using Azure.Messaging.EventHubs.Producer;
using System;
using System.Text;
using System.Threading.Tasks;

namespace EventHubsSample
{
    class Program
    {
        // Replace with your actual Event Hubs connection string
        // Or set it as an environment variable: EVENTHUB_CONNECTION_STRING
        private const string connectionString = "YOUR_EVENT_HUBS_CONNECTION_STRING";
        private const string eventHubName = "YOUR_EVENT_HUB_NAME"; // Optional, if not in connection string

        static async Task Main(string[] args)
        {
            Console.WriteLine("Azure Event Hubs Producer Sample");

            string envConnectionString = Environment.GetEnvironmentVariable("EVENTHUB_CONNECTION_STRING");
            string finalConnectionString = string.IsNullOrEmpty(connectionString) ? envConnectionString : connectionString;

            if (string.IsNullOrEmpty(finalConnectionString))
            {
                Console.ForegroundColor = ConsoleColor.Red;
                Console.WriteLine("Error: Event Hubs connection string is not configured.");
                Console.WriteLine("Please set the EVENTHUB_CONNECTION_STRING environment variable or update the 'connectionString' constant.");
                Console.ResetColor();
                return;
            }

            // Using a producer client that automatically polls and discovers partitions
            await using (var producer = new EventHubProducerClient(finalConnectionString, eventHubName))
            {
                try
                {
                    Console.WriteLine("Sending events...");
                    for (int i = 0; i < 10; i++)
                    {
                        var eventBody = $"{{ \"messageId\": {i}, \"timestamp\": \"{DateTime.UtcNow:O}\", \"content\": \"Hello Event Hubs! Event #{i}\" }}";
                        var eventData = new EventData(Encoding.UTF8.GetBytes(eventBody));

                        await producer.SendAsync(new[] { eventData });
                        Console.WriteLine($"Sent: {eventBody}");
                        await Task.Delay(100); // Small delay between events
                    }
                    Console.WriteLine("Finished sending events.");
                }
                catch (Exception ex)
                {
                    Console.ForegroundColor = ConsoleColor.Red;
                    Console.WriteLine($"An error occurred: {ex.Message}");
                    Console.ResetColor();
                }
            }

            Console.WriteLine("\nPress any key to exit...");
            Console.ReadKey();
        }
    }
}
Copied!

Note: Update "YOUR_EVENT_HUBS_CONNECTION_STRING" and optionally "YOUR_EVENT_HUB_NAME" if they are not set via environment variable.

Consumer Code

To run the consumer, you will need a separate project or modify Program.cs to handle consumption. For simplicity, let's create a new console app for the consumer. If you prefer, you can combine them, but separate projects are often cleaner.

1. Create a new .NET Console Application for the Consumer:

Command Line

cd ..
dotnet new console -n EventHubsConsumer
cd EventHubsConsumer
Copied!

2. Add the Azure Event Hubs SDK NuGet package:

Command Line

dotnet add package Azure.Messaging.EventHubs
Copied!

Replace the contents of EventHubsConsumer/Program.cs with the following consumer code:

using Azure.Messaging.EventHubs;
using Azure.Messaging.EventHubs.Consumer;
using System;
using System.Text;
using System.Threading.Tasks;

namespace EventHubsConsumer
{
    class Program
    {
        // Replace with your actual Event Hubs connection string
        // Or set it as an environment variable: EVENTHUB_CONNECTION_STRING
        private const string connectionString = "YOUR_EVENT_HUBS_CONNECTION_STRING";
        private const string eventHubName = "YOUR_EVENT_HUB_NAME"; // Optional, if not in connection string
        private const string consumerGroup = EventHubConsumerClient.DefaultConsumerGroupName; // Or specify your own consumer group

        static async Task Main(string[] args)
        {
            Console.WriteLine("Azure Event Hubs Consumer Sample");

            string envConnectionString = Environment.GetEnvironmentVariable("EVENTHUB_CONNECTION_STRING");
            string finalConnectionString = string.IsNullOrEmpty(connectionString) ? envConnectionString : connectionString;

            if (string.IsNullOrEmpty(finalConnectionString))
            {
                Console.ForegroundColor = ConsoleColor.Red;
                Console.WriteLine("Error: Event Hubs connection string is not configured.");
                Console.WriteLine("Please set the EVENTHUB_CONNECTION_STRING environment variable or update the 'connectionString' constant.");
                Console.ResetColor();
                return;
            }

            // Create a consumer client
            await using (var consumerClient = new EventHubConsumerClient(consumerGroup, finalConnectionString, eventHubName))
            {
                Console.WriteLine($"Listening to Event Hub: {eventHubName} in consumer group: {consumerGroup}");
                Console.WriteLine("Press Ctrl+C to stop.");

                try
                {
                    // Process events in batches
                    await foreach (PartitionEvent batch in consumerClient.ReadEventsAsync())
                    {
                        Console.WriteLine($"Received event from partition {batch.Partition.Id}:");
                        var messageBody = Encoding.UTF8.GetString(batch.Data.EventBody.ToArray());
                        Console.WriteLine($"  Body: {messageBody}");
                        Console.WriteLine($"  Sequence Number: {batch.Data.SequenceNumber}");
                        Console.WriteLine($"  Offset: {batch.Data.Offset}");
                        Console.WriteLine($"  Enqueued Time: {batch.Data.EnqueuedTime}");
                        Console.WriteLine("---");
                    }
                }
                catch (TaskCanceledException)
                {
                    Console.WriteLine("Consumer stopped by user.");
                }
                catch (Exception ex)
                {
                    Console.ForegroundColor = ConsoleColor.Red;
                    Console.WriteLine($"An error occurred: {ex.Message}");
                    Console.ResetColor();
                }
            }

            Console.WriteLine("\nPress any key to exit...");
            Console.ReadKey();
        }
    }
}
Copied!

Note: Update "YOUR_EVENT_HUBS_CONNECTION_STRING" and optionally "YOUR_EVENT_HUB_NAME" if they are not set via environment variable. Also, consider specifying a dedicated consumer group for your application.

Running the Sample

1. Set the Connection String:

Before running, ensure your Event Hubs connection string is available. You can set it as an environment variable:

Windows (Command Prompt)

set EVENTHUB_CONNECTION_STRING="YOUR_EVENT_HUBS_CONNECTION_STRING"
Copied!

Windows (PowerShell)

$env:EVENTHUB_CONNECTION_STRING="YOUR_EVENT_HUBS_CONNECTION_STRING"
Copied!

Linux/macOS

export EVENTHUB_CONNECTION_STRING="YOUR_EVENT_HUBS_CONNECTION_STRING"
Copied!

Replace "YOUR_EVENT_HUBS_CONNECTION_STRING" with your actual connection string.

2. Run the Producer:

Command Line

cd EventHubsSample
dotnet run
Copied!

You should see output indicating events being sent.

3. Run the Consumer:

In a separate terminal window, navigate to the consumer project directory and run it:

Command Line

cd ../EventHubsConsumer
dotnet run
Copied!

You should see output indicating events being received from the Event Hub.

Press Ctrl+C in the consumer's terminal to stop it.

Next Steps