Trigger Specification

This specification describes the azure-eventhub trigger for Azure Event Hubs.

triggers:
- type: azure-eventhub
  metadata:
    connectionFromEnv: EVENTHUB_CONNECTIONSTRING_ENV_NAME
    storageConnectionFromEnv: STORAGE_CONNECTIONSTRING_ENV_NAME
    consumerGroup: $Default
    unprocessedEventThreshold: '64'
    activationUnprocessedEventThreshold: '10'
    blobContainer: 'name_of_container'
    # Optional (Default: AzurePublicCloud)
    cloud: Private
    # Required when cloud = Private
    endpointSuffix: servicebus.airgap.example
    # Required when cloud = Private.
    activeDirectoryEndpoint: https://login.airgap.example/
    # Required when cloud = Private.
    eventHubResourceURL: https://eventhubs.airgap.example/

Parameter list:

• connectionFromEnv - Name of the environment variable your deployment uses to get the connection string appended with EntityPath=<event_hub_name>.
• storageConnectionFromEnv - Name of the environment variable that provides connection string for Azure Storage Account to store checkpoint. As of now the Event Hub scaler only reads from Azure Blob Storage.
• consumerGroup - Consumer group of Azure Event Hub consumer. (default: $default, Optional)
• unprocessedEventThreshold - Average target value to trigger scaling actions. (Default: 64, Optional)
• activationUnprocessedEventThreshold - Target value for activating the scaler. Learn more about activation here.(Default: 0, Optional)
• blobContainer - Container name to store checkpoint. This is needed for every checkpointStrategy except of AzureFunction. With Azure Functions the blobContainer is autogenerated and cannot be overridden.
• checkpointStrategy - configure the checkpoint behaviour of different Event Hub SDKs. (Values: azureFunction, blobMetadata, goSdk, default: "", Optional)
  • azureFunction - Suitable for Azure Functions & Azure WebJobs SDK. This is the default setting, when blobcontainer is not specified.
  • blobMetadata - For all implementations that store checkpoint information on blob metadata such as current C#, Python, Java and JavaScript Event Hub SDKs.
  • goSdk - For all implementations using the Golang SDK’s checkpointing, for example Dapr.
  • When no checkpoint strategy is specified, the Event Hub scaler will use backwards compatibility and able to scale older implementations of C#, Python or Java Event Hub SDKs. (see “Legacy checkpointing”). If this behaviour should be used, blobContainer is also required.
• cloud - Name of the cloud environment that the Event Hub belongs to. (Values: AzurePublicCloud, AzureUSGovernmentCloud, AzureChinaCloud, AzureGermanCloud, Private, Default: AzurePublicCloud, Optional)
• endpointSuffix - Service Bus endpoint suffix of the cloud environment. (Required when cloud is set to Private, e.g. servicebus.cloudapi.de for AzureGermanCloud).
• activeDirectoryEndpoint - Active Directory endpoint of the cloud environment. (Required when cloud is set to Private, e.g. https://login.microsoftonline.de/ for AzureGermanCloud).
• eventHubResourceURL - Event Hub resource URL of the cloud environment. (Required when cloud is set to Private, e.g. https://eventhubs.azure.net/ for known Azure Clouds).

💡 Learn more about the checkpointing behaviour in this section.

💡 The Azure Storage connection string is not compatible with connection string created from a Shared Access Signature.

Authentication Parameters

The common way of authenticating to Azure Event Hub is by using the connection string. However, you can use Azure AD Pod Identity or Azure AD Workload Identity providers if you host your cluster in Azure AKS, and if have configured it to support Pod Identity.

To use Pod Identity, you have to add a TriggerAuthentication and configure it to use Pod Identity like so:

apiVersion: keda.sh/v1alpha1
kind: TriggerAuthentication
metadata:
  name: nameOfTriggerAuth
  namespace: default
spec:
  podIdentity:
    provider: Azure | azure-workload

When you do so, the Event Hub scaler will depend on the existence of two configurations you have to provide: eventHubNamespace and eventHubName.

Checkpointing Behaviour

The list of available checkpointing strategies can be found in the trigger specification section. The way checkpoints are stored has changed with updates to the EventHub SDKs.

• Legacy behaviour: The older implementations are based on the EventProcessorHost client, which stores the checkpoint information as contents of the storage blob. This is the default behaviour when no checkpointStrategy is specified. This is applicable for the following scenarios:

  • .NET applications using Microsoft.Azure.EventHubs NuGet package.
  • Java applications using azure-eventhubs-eph package.
  • Python applications using azure-eventhub package below v5.

• Current behaviour: The newer implementations are based on the EventProcessorClient, which stores the checkpoint information as metadata on the storage blob. This is the behaviour when checkpointStrategy is set to blobMetadata. This is applicable for the following scenarios:

  • .NET applications using Azure.Messaging.EventHubs NuGet package.
  • Python applications using azure-eventhub v5.
  • .NET Azure Functions using Microsoft.Azure.WebJobs.Extensions.EventHubs v5.
  • Azure Functions in other languages using Microsoft.Azure.Functions.ExtensionBundle v3.

💡 blobContainer name is required for applications following legacy behaviour.

💡 Users should set blobContainer to azure-webjobs-eventhub for Azure Functions using blobMetadata as checkpointStrategy.

Example

apiVersion: keda.sh/v1alpha1
kind: ScaledObject
metadata:
  name: azure-eventhub-scaledobject
  namespace: default
spec:
  scaleTargetRef:
    name: azureeventhub-function
  triggers:
  - type: azure-eventhub
    metadata:
      # Required
      storageConnectionFromEnv: AzureWebJobsStorage
      # Required if not using Pod Identity
      connectionFromEnv: EventHub
      # Required if using Pod Identity
      eventHubNamespace: AzureEventHubNameSpace
      eventHubName: NameOfTheEventHub
# Optional
      consumerGroup: $Default # default: $Default
      unprocessedEventThreshold: '64' # default 64 events.
      blobContainer: ehcontainer