Kafka
Minimum Kafka ACLs for Nilus stream pipelines, across all four supported security protocols. Kafka is supported as a source only.
This page documents the minimum Kafka ACLs a platform or Kafka administrator must provision for Nilus to run a stream pipeline against a Kafka topic. Kafka is supported as a source only (type: stream); no write, CDC, or metadata pipeline mode exists for Kafka.
Supported security protocols
PLAINTEXT
Yes
No authentication
SASL_PLAINTEXT
Yes
SASL credentials over a plain (unencrypted) connection
SSL
Yes
TLS encryption with CA certificate verification (server cert only)
SASL_SSL
Yes
SASL credentials over a TLS-encrypted connection
Read data: Kafka as source (type: stream)
Nilus uses this path when the depot is referenced in source.address of a pipeline config. Nilus consumes messages from the target topic using the Kafka consumer protocol.
Kafka permissions are managed through ACLs, scoped to the principal User:<username> for SASL-authenticated connections. For PLAINTEXT, no ACLs are enforced.
Minimum grants
# Allow reading messages from the topic
kafka-acls.sh --bootstrap-server <broker> \
--add --allow-principal User:<username> \
--operation Read --topic <topic_name>
# Allow committing and reading consumer offsets
kafka-acls.sh --bootstrap-server <broker> \
--add --allow-principal User:<username> \
--operation Read --group '*'
# Allow topic metadata lookup (required for partition assignment)
kafka-acls.sh --bootstrap-server <broker> \
--add --allow-principal User:<username> \
--operation Describe --topic <topic_name>READ
Topic:<topic_name>
Consume messages from the topic
READ
Group:<pipeline_resource_id>
Commit and read consumer offsets
DESCRIBE
Topic:<topic_name>
Fetch partition metadata for consumer assignment
Permission matrix
Use case
PLAINTEXT
SASL_PLAINTEXT
SSL
SASL_SSL
Kafka ACL required
Stream read (topic consumer)
Yes
Yes
Yes
Yes
READ topic + READ group + DESCRIBE topic
Notes
Kafka is a source-only connector in Nilus. There is no write, CDC, or metadata pipeline mode for Kafka depots.
ACL enforcement depends on the security protocol:
PLAINTEXTconnections are not subject to ACL checks, whileSASL_PLAINTEXT,SSL, andSASL_SSLall require the ACLs above.The consumer group ACL targets
Group:<pipeline_resource_id>. GrantREADon the specific group your pipeline resource uses, or scope it to'*'if the group name is not known ahead of time.
Last updated
Was this helpful?