> For the complete documentation index, see [llms.txt](https://v2.dataos.info/llms.txt). Markdown versions of documentation pages are available by appending `.md` to page URLs; this page is available as [Markdown](https://v2.dataos.info/references/v1/resources/nilus/roles-and-permissions/kafka-minimum-permissions.md).

# Kafka

This page documents the minimum Kafka ACLs a platform or Kafka administrator must provision for Nilus to run a stream pipeline against a Kafka topic. Kafka is supported as a **source only** (`type: stream`); no write, CDC, or metadata pipeline mode exists for Kafka.

## Supported security protocols

| Protocol         | Supported | Auth method                                                        |
| ---------------- | --------- | ------------------------------------------------------------------ |
| `PLAINTEXT`      | Yes       | No authentication                                                  |
| `SASL_PLAINTEXT` | Yes       | SASL credentials over a plain (unencrypted) connection             |
| `SSL`            | Yes       | TLS encryption with CA certificate verification (server cert only) |
| `SASL_SSL`       | Yes       | SASL credentials over a TLS-encrypted connection                   |

## Read data: Kafka as source (`type: stream`)

Nilus uses this path when the depot is referenced in `source.address` of a pipeline config. Nilus consumes messages from the target topic using the Kafka consumer protocol.

Kafka permissions are managed through ACLs, scoped to the principal `User:<username>` for SASL-authenticated connections. For `PLAINTEXT`, no ACLs are enforced.

### Minimum grants

```bash
# Allow reading messages from the topic
kafka-acls.sh --bootstrap-server <broker> \
  --add --allow-principal User:<username> \
  --operation Read --topic <topic_name>

# Allow committing and reading consumer offsets
kafka-acls.sh --bootstrap-server <broker> \
  --add --allow-principal User:<username> \
  --operation Read --group '*'

# Allow topic metadata lookup (required for partition assignment)
kafka-acls.sh --bootstrap-server <broker> \
  --add --allow-principal User:<username> \
  --operation Describe --topic <topic_name>
```

| ACL        | Resource type                  | Why required                                     |
| ---------- | ------------------------------ | ------------------------------------------------ |
| `READ`     | `Topic:<topic_name>`           | Consume messages from the topic                  |
| `READ`     | `Group:<pipeline_resource_id>` | Commit and read consumer offsets                 |
| `DESCRIBE` | `Topic:<topic_name>`           | Fetch partition metadata for consumer assignment |

## Permission matrix

| Use case                     | `PLAINTEXT` | `SASL_PLAINTEXT` | `SSL` | `SASL_SSL` | Kafka ACL required                             |
| ---------------------------- | ----------- | ---------------- | ----- | ---------- | ---------------------------------------------- |
| Stream read (topic consumer) | Yes         | Yes              | Yes   | Yes        | `READ` topic + `READ` group + `DESCRIBE` topic |

## Notes

* Kafka is a source-only connector in Nilus. There is no write, CDC, or metadata pipeline mode for Kafka depots.
* ACL enforcement depends on the security protocol: `PLAINTEXT` connections are not subject to ACL checks, while `SASL_PLAINTEXT`, `SSL`, and `SASL_SSL` all require the ACLs above.
* The consumer group ACL targets `Group:<pipeline_resource_id>`. Grant `READ` on the specific group your pipeline resource uses, or scope it to `'*'` if the group name is not known ahead of time.


---

# Agent Instructions
This documentation is published with GitBook. GitBook is the documentation platform designed so that both humans and AI agents can read, navigate, and reason over technical content effectively. Learn more at gitbook.com.

## Querying This Documentation
If you need additional information that is not directly available in this page, you can query the documentation dynamically by asking a question.

Perform an HTTP GET request on the current page URL with the `ask` query parameter, and the optional `goal` query parameter:

```
GET https://v2.dataos.info/references/v1/resources/nilus/roles-and-permissions/kafka-minimum-permissions.md?ask=<question>&goal=<endgoal>
```

`ask` is the immediate question: it should be specific, self-contained, and written in natural language.
`goal` is optional and describes the broader end goal you are ultimately trying to accomplish on behalf of the user. GitBook uses it to tailor the answer towards what is most useful for that goal.

The response will contain a direct answer to the question and relevant excerpts and sources from the documentation.

Use this mechanism when the answer is not explicitly present in the current page, you need clarification or additional context, or you want to retrieve related documentation sections.