> For the complete documentation index, see [llms.txt](https://v2.dataos.info/llms.txt). Markdown versions of documentation pages are available by appending `.md` to page URLs; this page is available as [Markdown](https://v2.dataos.info/concepts/resources/secret/consume.md). # Consume Users can consume a Secret in DataOS by referencing it to the Resource that requires access to data sources. The most common patterns are through a Depot, directly in Nilus, and in Vulcan repository sync configuration. {% hint style="info" %} A Secret created in one Tenant can also be used in another Tenant within the same DataOS environment, if access is granted explicitly. For example, a Data Admin can create a PostgreSQL Secret in the `engineering` Tenant and allow users in the `product` Tenant to consume the same Secret. This access can be granted by a Data Admin, Operator, or Tenant Admin to users of the target Tenant. {% endhint %} ## Consume Secret in Depot Depot uses `spec.secrets` to attach one or more Secret Resources. ```yaml name: snowflake-depot version: v2alpha type: depot layer: user spec: type: snowflake spec: warehouse: COMPUTE_WH url: abc12345.snowflakecomputing.com database: ANALYTICS account: abc12345 schema: PUBLIC secrets: - id: engineering:snowflake-secret-r purpose: scan - id: engineering:snowflake-secret-rw purpose: rw ``` ## Consume Secret in Nilus [Nilus](/concepts/resources/nilus.md) supports three Secret resolution patterns. Choose the pattern based on how the source, sink, or repo is defined. ### How Nilus resolves Secrets? 1. Use explicit projection for connectors configured directly in the URI. 2. Use automatic inference for `dataos://` source and sink addresses. 3. Use repo Secret inference for `stackSpec.repo.secretId`. Use this rule: * `dataos://` means depot credentials are inferred. * `{ENV_VAR}` means you must project the value explicitly. * `repo.secretId` means git-sync credentials are inferred. ### Pattern 1: Explicit projection for non-depot connectors Use this when the connector is configured directly in the address. This applies when the address contains placeholders such as `{USERNAME}`, `{PASSWORD}`, `{TOKEN}`, or `{CREDENTIALS_B64}`. Explicit projection relies on the DataOS Projections templating system to inject Secret values into the workload at runtime. See [Projections](/concepts/resources/secret/projections.md) for the full template syntax, data sources, and available variables. Common cases: * API-style connectors * SaaS connectors with token auth * connectors that expect username and password in the URI * connectors that expect an encoded credential payload #### How it works? 1. Select the Secret in `use.projection.secrets`. 2. Give it a `contextAlias`. 3. Map required fields to env vars in `use.projection.projections.envVars`. 4. Reference those env vars in the connector address. #### Example: Username and password projection ```yaml name: connector-to-pg-test version: v2alpha type: workflow workflow: dag: - name: sync spec: stack: nilus:1.0 use: projection: secrets: - id: tenant:connector-secret contextAlias: connectorsecret projections: envVars: - key: CONNECTOR_USERNAME template: "{{ secrets['connectorsecret'].USERNAME | base64_decode }}" - key: CONNECTOR_PASSWORD template: "{{ secrets['connectorsecret'].PASSWORD | base64_decode }}" stackSpec: source: address: connector://?username={CONNECTOR_USERNAME}&password={CONNECTOR_PASSWORD} options: source-table: "resource_name" sink: address: dataos://niluspgdepot?purpose=rw options: dest-table: "testing_nilus.connector_to_pg" incremental-strategy: append ``` #### Example: Encoded credential projection ```yaml name: encoded-credential-test version: v2alpha type: workflow workflow: dag: - name: sync spec: stack: nilus:1.0 use: projection: secrets: - id: tenant:file-secret contextAlias: connectorsecret projections: envVars: - key: CREDENTIALS_B64 template: "{{ secrets['connectorsecret'].json_key_file }}" stackSpec: source: address: connector://?credentials_base64={CREDENTIALS_B64} options: source-table: "resource_name" ``` #### Notes * Project only the fields the connector needs. * Use clear env var names such as `API_TOKEN` or `CREDENTIALS_B64`. * Nilus resolves the placeholders at runtime. ### Pattern 2: Automatic Depot Secret inference If `source.address` or `sink.address` starts with `dataos://`, Nilus treats it as a Depot reference. The Stack projects the matching Depot spec and Secret automatically. In this pattern, you do not assemble the source or sink credentials manually. #### Example ```yaml name: postgres-to-lakehouse version: v2alpha type: workflow workflow: dag: - name: sync-orders spec: stack: nilus:1.0 stackSpec: source: address: dataos://production_postgres?purpose=read options: source-table: "sales.orders" sink: address: dataos://analytics_lakehouse?purpose=rw options: dest-table: "analytics.orders_raw" incremental-strategy: append ``` #### How `purpose` works? Set `purpose` explicitly when the Depot supports multiple access modes. This lets Nilus select the correct purpose-specific Secret. Examples: * `dataos://source_depot?purpose=read` * `dataos://sink_depot?purpose=rw` #### What happens at runtime Nilus reads the projected Depot files from the runtime and builds the final connector configuration. This is why `dataos://` behaves differently from direct URIs that use `{ENV_VAR}` placeholders. ### Pattern 3: Repo Secret inference Nilus also supports one stack-managed repo, Secret path through `stackSpec.repo.secretId`. This is used for Git-sync authentication with repo-backed custom sources or destinations. #### Example ```yaml stackSpec: repo: url: https://bitbucket.org/org/custom-connectors baseDir: connectors secretId: repo-sync-secret ``` In this pattern, the stack wires `GITSYNC_USERNAME` and `GITSYNC_PASSWORD` automatically for git-sync. ### Best practices * Use `dataos://` when the source or sink is backed by a Depot. * Use explicit projection when the connector URI contains credential placeholders. * Use `repo.secretId` only for repo sync authentication. ### Common mistakes * Expecting direct connector credentials to be inferred automatically. * Not to project env vars for non-`dataos://` connectors. * Not adding `purpose` on `dataos://` addresses when multiple purposes exist. * Adding direct connector credentials with `dataos://` when it is not supported. ## Consume Secret in Vulcan ### Pattern 1: Vulcan data access via Depot [Vulcan](https://tmdc-io.github.io/vulcan-book/guides/get-started/docker/) accesses source systems via Depot, so the Secret is consumed through the Depot reference. ```yaml gateways: default: connection: type: depot address: dataos://snowflake-depot ``` ### Pattern 2: Vulcan repository authentication [Vulcan](https://tmdc-io.github.io/vulcan-book/guides/get-started/docker/) uses a Secret for Git repository sync in `domain-resource.yaml`. Use `workspace:secret-name` format: ```yaml spec: repo: url: https://bitbucket.org/your-org/your-repo syncFlags: - '--ref=main' - '--submodules=off' baseDir: vulcan-project secretId: engineering:git-sync ``` ## Notes * Ensure the Secret exists before applying Depot/Nilus/Vulcan manifests. * Prefer Depot-based secret handling wherever supported. --- # Agent Instructions This documentation is published with GitBook. GitBook is the documentation platform designed so that both humans and AI agents can read, navigate, and reason over technical content effectively. Learn more at gitbook.com. ## Querying This Documentation If you need additional information that is not directly available in this page, you can query the documentation dynamically by asking a question. Perform an HTTP GET request on the current page URL with the `ask` query parameter, and the optional `goal` query parameter: ``` GET https://v2.dataos.info/concepts/resources/secret/consume.md?ask=&goal= ``` `ask` is the immediate question: it should be specific, self-contained, and written in natural language. `goal` is optional and describes the broader end goal you are ultimately trying to accomplish on behalf of the user. GitBook uses it to tailor the answer towards what is most useful for that goal. The response will contain a direct answer to the question and relevant excerpts and sources from the documentation. Use this mechanism when the answer is not explicitly present in the current page, you need clarification or additional context, or you want to retrieve related documentation sections.