Azure Functions Input Bindings: Event Hubs

Introduction to Event Hubs Input Bindings

Azure Event Hubs is a highly scalable data streaming platform and event ingestion service. Azure Functions can integrate with Event Hubs using input bindings to easily consume events from an Event Hub. This allows your functions to react to incoming data streams without managing low-level client code.

Input bindings enable your function to read data from an Event Hub as a trigger or as a parameter to your function. This documentation focuses on using Event Hubs as an input binding to retrieve messages.

How Event Hubs Input Bindings Work

When configured as an input binding, Azure Functions automatically reads messages from a specified Event Hub partition. The binding can be configured to receive a single message, a batch of messages, or even specific properties of the Event Hub message.

The runtime handles checkpointing for you, ensuring that messages are processed reliably. You typically don't need to worry about offset management when using the input binding.

Configuration

The Event Hubs input binding is configured in your function.json file. Here's a typical configuration for an input binding:

{
  "bindings": [
    {
      "type": "eventHubTrigger",
      "name": "messages",
      "direction": "in",
      "eventHubName": "my-event-hub",
      "connection": "EventHubConnectionAppSetting"
    }
    // ... other bindings
  ]
}

Properties:

The connection property refers to an application setting in your Azure Functions configuration, typically containing a connection string like: Endpoint=sb://your-event-hubs-namespace.servicebus.windows.net/;SharedAccessKeyName=your-key-name;SharedAccessKey=your-key-value

Function Code Examples

The name property in function.json maps to the parameter name in your function.

C# Example (Single Message)

When cardinality is not specified or is one, you receive a single event.

using Microsoft.Azure.EventHubs;
using Microsoft.Azure.WebJobs;
using Microsoft.Extensions.Logging;

public static class EventHubInputFunction
{
    [FunctionName("ProcessEventHubMessage")]
    public static void Run(
        [EventHubTrigger("my-event-hub", Connection = "EventHubConnectionAppSetting")] EventData message,
        ILogger log)
    {
        log.LogInformation($"C# Event Hub trigger function processed a message: {message.SystemProperties.SequenceNumber}");
        log.LogInformation($"Message body: {System.Text.Encoding.UTF8.GetString(message.Body)}");
    }
}

C# Example (Batch of Messages)

To receive a batch of messages, set cardinality to many in function.json.

using Microsoft.Azure.EventHubs;
using Microsoft.Azure.WebJobs;
using Microsoft.Extensions.Logging;
using System.Collections.Generic;

public static class EventHubBatchInputFunction
{
    [FunctionName("ProcessEventHubMessages")]
    public static void Run(
        [EventHubTrigger("my-event-hub", Connection = "EventHubConnectionAppSetting", Cardinality = Cardinality.Many)] EventData[] messages,
        ILogger log)
    {
        log.LogInformation($"C# Event Hub trigger function processed a batch of {messages.Length} messages.");

        foreach (var message in messages)
        {
            log.LogInformation($"Message SequenceNumber: {message.SystemProperties.SequenceNumber}");
            log.LogInformation($"Message body: {System.Text.Encoding.UTF8.GetString(message.Body)}");
        }
    }
}

Node.js Example (Batch of Messages)

In Node.js, a batch is represented as an array.

module.exports = async function (context, messages) {
    context.log(`JavaScript event hub trigger function processed a batch of ${messages.length} messages.`);

    for (let i = 0; i < messages.length; i++) {
        const message = messages[i];
        context.log(`Processing message: ${message.sequenceNumber}`);
        context.log(`Message body: ${message.body}`);
    }
};

And the corresponding function.json for Node.js (assuming batch):

{
  "bindings": [
    {
      "type": "eventHubTrigger",
      "name": "messages",
      "direction": "in",
      "eventHubName": "my-event-hub",
      "connection": "EventHubConnectionAppSetting",
      "cardinality": "many"
    }
  ]
}

EventData Object

The EventData object (or its equivalent in other languages) provides access to the message content and metadata. Key properties include:

• Body: The message payload as a byte array.
• SystemProperties: Contains metadata like SequenceNumber, Offset, EnqueuedTimeUtc.
• Properties: Custom application properties added to the event.
• PartitionKey: The partition key used when sending the event.

Error Handling and Retries

Azure Functions runtime has built-in retry mechanisms for Event Hubs triggers. If your function fails to process a message (or batch), the runtime will attempt to redeliver it. For persistent failures, consider implementing dead-lettering mechanisms by sending problematic messages to a separate queue or storage.

Advanced Scenarios

• Partition Management: While the binding handles checkpointing, understanding partitions can help optimize processing and ensure ordering within a partition.
• Consumer Groups: Using dedicated consumer groups for different applications consuming from the same Event Hub is a best practice.
• Schema Evolution: If your event schema changes, ensure your function code can handle the variations or implement schema validation.