> For the complete documentation index, see [llms.txt](https://v2.dataos.info/llms.txt). Markdown versions of documentation pages are available by appending `.md` to page URLs; this page is available as [Markdown](https://v2.dataos.info/concepts/resources/depot/supported-sources/kafka.md). # Kafka To create a Kafka Depot, gather the following details. ## Pre-requisites specific to DataOS To create a Depot, you need the Data Admin role. Contact the DataOS Operator or Tenant Admin to get it. Run this command to see your assigned roles. ```bash dataos-ctl user get #Expected Output: time="2026-03-30T17:49:18+05:30" level=info msg="😃 user get..." time="2026-03-30T17:49:19+05:30" level=info msg="😃 user get...complete" NAME │ ID │ TYPE │ EMAIL │ TAGS ──────────────┼───────────────────┼────────┼──────────────────────┼──────────────────────────────────────── I Am Groot │ iamgroottmdcio │ person │ iam.groot@tmdc.io │ roles:id:ct-onboarding-data-developer, │ │ │ │ roles:id:data-dev, │ │ │ │ roles:id:system-dev, │ │ │ │ roles:id:user, │ │ │ │ users:id:iamgroottmdcio ``` ## Pre-requisites specific to the source system * **Kafka broker list**: Comma-separated broker addresses (for example, `broker1:9092,broker2:9092`). * **Schema Registry URL (optional)**: Required for schema-based formats such as Avro. * **Security/authentication mode**: Choose one of the supported Kafka connection modes shown below. ## Create a Kafka Depot DataOS supports multiple Kafka security modes. The Depot manifest stays the same shape. The referenced Secret changes by mode. ### Supported Kafka connection modes | Mode | `security_protocol` | `sasl_mechanism` | Typical Secret fields | | ---------------------- | ------------------- | ---------------- | ------------------------------------------------------- | | Passwordless | `PLAINTEXT` | NA | `security_protocol` | | SSL | `SSL` | NA | `trust_store_*`, `key_store_*`, `key_password`, `files` | | SASL + PLAINTEXT | `SASL_PLAINTEXT` | `PLAIN` | `username`, `password` | | SASL + SSL (PLAIN) | `SASL_SSL` | `PLAIN` | `trust_store_*`, `username`, `password`, `files` | | SASL + SSL (SCRAM-256) | `SASL_SSL` | `SCRAM-SHA-256` | `trust_store_*`, `username`, `password`, `files` | Follow the steps below to create a Kafka Depot. ### **Step 1: Create a Secret for securing Kafka credentials** Create a Kafka Secret based on your selected mode by following the [Kafka Secret](/concepts/resources/secret/data-sources/kafka.md) guide. ### **Step 2: Create a Kafka Depot manifest file** Create a manifest file for your Kafka Depot. {% tabs %} {% tab title="Manifest file" %} ```yaml name: ${{passwordless-name}} version: v2alpha type: depot tags: - Kafka - depot - streaming description: "Apache Kafka streaming platform depot" spec: type: kafka spec: brokers: - ${{passwordless-broker-name}} schemaRegistryUrl: ${{passwordless-schemaregistry-url}} secrets: - id: "${{tenant-id}}:${{passwordless-secret-name}}" purpose: rw - id: "${{tenant-id}}:${{passwordless-secret-name}}" purpose: scan - id: "${{tenant-id}}:${{passwordless-secret-name}}" purpose: query ``` {% endtab %} {% tab title="Example" %} ```yaml name: ${{passwordless-name}} version: v2alpha type: depot tags: - Kafka - depot - streaming description: "Apache Kafka streaming platform depot" spec: type: kafka spec: brokers: - ${{passwordless-broker-name}} schemaRegistryUrl: ${{passwordless-schemaregistry-url}} secrets: - id: "${{tenant-id}}:${{passwordless-secret-name}}" purpose: rw - id: "${{tenant-id}}:${{passwordless-secret-name}}" purpose: scan - id: "${{tenant-id}}:${{passwordless-secret-name}}" purpose: query ``` {% endtab %} {% endtabs %} The Depot manifest stays the same for all Kafka modes. Only the Secret values change, based on the selected `security_protocol` and `sasl_mechanism`. ### **Step 3: Apply the Depot manifest file** Apply the manifest with the DataOS CLI: ```bash dataos-ctl resource apply -f ${{manifest-file-path}} ``` ## Verify the Depot creation Verify the Depot in two ways: * List Depots where you are the owner: ```bash dataos-ctl resource get -t depot ``` * List all Depots in the current Tenant: ```bash dataos-ctl resource get -t depot -a ``` ## Delete a Depot {% hint style="warning" %} Best practice: Delete Resources that are no longer in use to save time and reduce costs. {% endhint %} To delete a Depot, use the DataOS CLI: {% tabs %} {% tab title="Command 1" %} ```bash dataos-ctl resource delete -t depot -n ${{name}} ``` {% endtab %} {% tab title="Command 2 " %} ```bash dataos-ctl resource delete -i "${{resource-name}}|v2alpha|depot" ``` {% endtab %} {% tab title="Command 3" %} ```bash dataos-ctl resource delete -f ${{manifest-file-path}} ``` {% endtab %} {% endtabs %} Specify the Resource type and Depot name in the `delete` command. **Example:** {% tabs %} {% tab title="Command 1" %} ```bash dataos-ctl resource delete -t depot -n testdepot #output time="2026-03-25T15:53:55+05:30" level=info msg="🗑 delete..." time="2026-03-25T15:53:55+05:30" level=info msg="🗑 deleting testdepot:v2alpha:depot..." time="2026-03-25T15:53:56+05:30" level=info msg="🗑 deleting testdepot:v2alpha:depot...deleted" time="2026-03-25T15:53:56+05:30" level=info msg="🗑 delete...complete" time="2026-03-25T15:53:56+05:30" level=info msg="🗑 delete...complete" ``` {% endtab %} {% tab title="Command 2" %} ```bash dataos-ctl resource delete -i "testdepot|v2alpha|depot" #output time="2026-03-25T15:55:37+05:30" level=info msg="🗑 delete..." time="2026-03-25T15:55:37+05:30" level=info msg="🗑 deleting testdepot:v2alpha:depot..." time="2026-03-25T15:55:37+05:30" level=info msg="🗑 deleting testdepot:v2alpha:depot...deleted" time="2026-03-25T15:55:37+05:30" level=info msg="🗑 delete...complete" ``` {% endtab %} {% tab title="Command 3" %} ```bash dataos-ctl resource delete -f /path/to/depot.yaml #output time="2026-03-25T15:53:55+05:30" level=info msg="🗑 delete..." time="2026-03-25T15:53:55+05:30" level=info msg="🗑 deleting testdepot:v2alpha:depot..." time="2026-03-25T15:53:56+05:30" level=info msg="🗑 deleting testdepot:v2alpha:depot...deleted" time="2026-03-25T15:53:56+05:30" level=info msg="🗑 delete...complete" ``` {% endtab %} {% endtabs %} --- # Agent Instructions This documentation is published with GitBook. GitBook is the documentation platform designed so that both humans and AI agents can read, navigate, and reason over technical content effectively. Learn more at gitbook.com. ## Querying This Documentation If you need additional information that is not directly available in this page, you can query the documentation dynamically by asking a question. Perform an HTTP GET request on the current page URL with the `ask` query parameter, and the optional `goal` query parameter: ``` GET https://v2.dataos.info/concepts/resources/depot/supported-sources/kafka.md?ask=&goal= ``` `ask` is the immediate question: it should be specific, self-contained, and written in natural language. `goal` is optional and describes the broader end goal you are ultimately trying to accomplish on behalf of the user. GitBook uses it to tailor the answer towards what is most useful for that goal. The response will contain a direct answer to the question and relevant excerpts and sources from the documentation. Use this mechanism when the answer is not explicitly present in the current page, you need clarification or additional context, or you want to retrieve related documentation sections.