Kafka Triggers

Orvanta can connect to Kafka broker servers and trigger runnables (scripts, flows) when a message is received. This functionality is available as a self-hosted Enterprise feature.

Resource configuration

Before creating a Kafka trigger, you must establish a Kafka resource with:

• Brokers: List of broker hostnames in hostname:port format.
• Security: Authentication and encryption settings.

Security modes

The platform supports seven security configurations:

Mode	Description
PLAINTEXT	No authentication or encryption (development only)
SASL_PLAINTEXT	Username/password without encryption (PLAIN, SCRAM-SHA-256, SCRAM-SHA-512)
SSL	TLS encryption with optional client certificate
SASL_SSL	Username/password with TLS encryption
SASL_GSSAPI	Kerberos (GSSAPI) without encryption
SASL_SSL_GSSAPI	Kerberos with TLS encryption
SASL_SSL_OAUTHBEARER	OAuth 2.0/OIDC with TLS encryption

Kerberos authentication

Required properties for SASL_GSSAPI or SASL_SSL_GSSAPI:

• kerberos_service_name: Kerberos principal name (default: kafka).
• kerberos_principal: Client’s principal (e.g., user@REALM).
• keytab_path or keytab_base64: Keytab credentials (one required).

For non-root servers, the keytab file must be readable by that user. Use keytab_base64 or adjust volume mount permissions in Kubernetes with defaultMode: 0644.

OAuth 2.0/OIDC authentication

Required properties for SASL_SSL_OAUTHBEARER:

• client_id: OAuth client identifier.
• client_secret: OAuth client secret.
• token_endpoint_url: IdP token endpoint URL.
• scope: Optional space-separated OAuth scopes.
• extensions: Optional comma-separated key=value pairs.

Additional TLS certificate options are available for mutual authentication.

Usage configuration

Initial offset

Controls where the consumer starts reading when the group has no committed offset:

• Latest (default): Only processes new messages.
• Earliest: Processes all existing messages from the beginning.

Auto-commit offsets

By default, Orvanta commits offsets once the job has been successfully pushed to the queue. Disabling auto-commit requires manual offset management via SDK helpers, with offsets committed every 5 seconds per topic/partition pair.

Reset offset

The Reset offset to earliest button restarts the consumer from the beginning, reprocessing all messages.

Runnable configuration

The received message payload is base64-encoded and passed as a string argument. A preprocessor can decode and structure the data for the main function.

Example preprocessor structure:

event: {
  kind: "kafka";
  trigger_path: string;
  payload: string; // base64 encoded
  brokers: string[];
  topic: string;
  group_id: string;
  partition: number;
  offset: number;
}

Filters

Message filtering evaluates the decoded JSON payload before triggering. Filters use key paths (e.g., type, data.status) with expected values.

Multiple filters can use:

• AND (default): Message must match all filters.
• OR: Message triggers on any filter match.

The matching uses a superset check — the message value at the given key must contain all fields from the filter value.

Error handling

Kafka triggers support local error handlers that override workspace error handlers for specific triggers.