Skip to content
For the complete documentation index, see llms.txt. This page is also available as Markdown.

Kafka

Create a Depot resource for Kafka topics and brokers across multiple security modes.

To create a Kafka Depot, gather the following details.

Pre-requisites specific to DataOS

A user with the Tenant Admin, Data Admin, or Data Developer Tenant role can create a Depot.

See Depot: Access control for who can create a Depot and how access to it is managed.

Run this command to see your assigned roles.

dataos-ctl user get
#Expected Output: 
time="2026-03-30T17:49:18+05:30" level=info msg="😃 user get..."
time="2026-03-30T17:49:19+05:30" level=info msg="😃 user get...complete"

     NAME     │        ID         │  TYPE  │        EMAIL         │                  TAGS
──────────────┼───────────────────┼────────┼──────────────────────┼────────────────────────────────────────
 I Am Groot   │ iamgroottmdcio    │ person │ iam.groot@tmdc.io    │ roles:id:ct-onboarding-data-developer,
              │                   │        │                      │ roles:id:user,
              │                   │        │                      │ users:id:iamgroottmdcio

Pre-requisites specific to the source system

  • Kafka broker list: Comma-separated broker addresses (for example, broker1:9092,broker2:9092).

  • Schema Registry URL (optional): Required for schema-based formats such as Avro.

  • Security/authentication mode: Choose one of the supported Kafka connection modes shown below.

Create a Kafka Depot

DataOS supports multiple Kafka security modes. The Depot manifest stays the same shape. The referenced Secret changes by mode.

Supported Kafka connection modes

Mode

security_protocol

sasl_mechanism

Typical Secret fields

Passwordless

PLAINTEXT

NA

security_protocol

SSL

SSL

NA

trust_store_*, key_store_*, key_password, files

SASL + PLAINTEXT

SASL_PLAINTEXT

PLAIN

username, password

SASL + SSL (PLAIN)

SASL_SSL

PLAIN

trust_store_*, username, password, files

SASL + SSL (SCRAM-256)

SASL_SSL

SCRAM-SHA-256

trust_store_*, username, password, files

Follow the steps below to create a Kafka Depot.

Step 1: Create a Secret for securing Kafka credentials

Create a Kafka Secret based on your selected mode by following the Kafka Secret guide.

Step 2: Create a Kafka Depot manifest file

Create a manifest file for your Kafka Depot.

The Depot manifest stays the same for all Kafka modes. Only the Secret values change, based on the selected security_protocol and sasl_mechanism.

Step 3: Apply the Depot manifest file

Apply the manifest with the DataOS CLI:

Verify the Depot creation

Verify the Depot in two ways:

  • List Depots where you are the owner:

  • List all Depots in the current Tenant:

Delete a Depot

To delete a Depot, use the DataOS CLI:

Specify the Resource type and Depot name in the delete command.

Example:

Last updated

Was this helpful?