> For the complete documentation index, see [llms.txt](https://v2.dataos.info/llms.txt). Markdown versions of documentation pages are available by appending `.md` to page URLs; this page is available as [Markdown](https://v2.dataos.info/references/dataos-resources/nilus/concepts/secrets-and-projections.md).

# Secrets and projections

> This guide explains how secrets are resolved in current `nilus` resources.

## TL;DR

Three secret-handling patterns in Nilus:

1. Explicit projection under `spec.use.projection` for direct connector URIs.
2. Automatic Depot inference for `dataos://` source and sink addresses.
3. Automatic repo-auth projection for `spec.repo.secretId`.

The shortest rule of thumb:

* `dataos://...purpose=` means Nilus infers the Depot connection.
* `{ENV_VAR}` inside a direct URI means you must project that value yourself.
* `spec.repo.secretId` is only for repo-backed extension sync.

## Mental model

Nilus secret handling has two layers:

1. The Nilus domain template decides which secrets or Depots must be projected into the runtime.
2. The Nilus runtime consumes those projected values to build the final source and sink configuration.

Separate two questions:

1. How does the stack project the values?
2. How does Nilus consume those values in the authored resource?

## Pattern 1: explicit projection for direct connector URIs

Use explicit projection when the source or sink address contains placeholders such as `{USERNAME}`, `{PASSWORD}`, `{TOKEN}`, or `{CREDENTIALS_B64}`. This is the right pattern for connectors configured directly in the URI rather than inferred from a `dataos://` Depot. Typical cases:

* API-style connectors configured directly in the URI.
* Connectors that expect username/password values in the address.
* Connectors that expect token-based values in the address.
* Connectors that expect a base64 or file-derived credential blob.

### How it works

Declare the secret under `spec.use.projection.secrets`, expose selected values through `spec.use.projection.projections.envVars`, then reference those values inside the authored URI.

```yaml
spec:
  use:
    projection:
      secrets:
        - id: tenant:my-connector-secret
          contextAlias: connectorSecret
      projections:
        envVars:
          - key: CONNECTOR_USERNAME
            template: "{{ secrets['connectorSecret'].USERNAME | base64_decode }}"
          - key: CONNECTOR_PASSWORD
            template: "{{ secrets['connectorSecret'].PASSWORD | base64_decode }}"
```

### Example: username / password / token projection

```yaml
name: connector-to-warehouse
version: v1alpha
type: nilus
spec:
  type: batch
  compute: universe-compute
  use:
    projection:
      secrets:
        - id: tenant:my-connector-secret
          contextAlias: connectorSecret
      projections:
        envVars:
          - key: CONNECTOR_USERNAME
            template: "{{ secrets['connectorSecret'].USERNAME | base64_decode }}"
          - key: CONNECTOR_PASSWORD
            template: "{{ secrets['connectorSecret'].PASSWORD | base64_decode }}"
          - key: CONNECTOR_TOKEN
            template: "{{ secrets['connectorSecret'].TOKEN | base64_decode }}"
          - key: CONNECTOR_DOMAIN
            template: "{{ secrets['connectorSecret'].DOMAIN | base64_decode }}"
  source:
    address: connector://?username={CONNECTOR_USERNAME}&password={CONNECTOR_PASSWORD}&token={CONNECTOR_TOKEN}&domain={CONNECTOR_DOMAIN}
    options:
      source_table: resource_name
  sink:
    address: dataos://niluspgdepot?purpose=rw
    options:
      dest_table: testing_nilus.connector_to_pg
      incremental_strategy: append
```

### Example: encoded credential projection

```yaml
name: encoded-credential-example
version: v1alpha
type: nilus
spec:
  type: batch
  compute: universe-compute
  use:
    projection:
      secrets:
        - id: tenant:my-file-secret
          contextAlias: connectorSecret
      projections:
        envVars:
          - key: CREDENTIALS_B64
            template: "{{ secrets['connectorSecret'].json_key_file }}"
  source:
    address: connector://?credentials_base64={CREDENTIALS_B64}
    options:
      source_table: resource_name
  sink:
    address: dataos://niluspgdepot?purpose=rw
    options:
      dest_table: testing_nilus.encoded_connector_to_pg
      incremental_strategy: append
```

### Projection rules

1. Put secret selection under `spec.use.projection.secrets`.
2. Expose only the fields the connector needs.
3. Reference those values through `{ENV_VAR}` placeholders in the authored URI.
4. Match the secret shape to the connector contract.
5. Prefer clear env-var names such as `API_TOKEN`, `CONNECTOR_USERNAME`, or `CREDENTIALS_B64`.

## Pattern 2: automatic Depot inference for `dataos://`

If `spec.source.address` or `spec.sink.address` starts with `dataos://`, Nilus treats that address as a Depot reference and projects the matching Depot metadata and secret into the runtime automatically. This is the simplest path when a connector should be resolved from a Depot rather than assembled manually.

### How `purpose` works

`purpose` is required on every `dataos://` Depot address. Always add it, even when the Depot defines only a single secret purpose. Nilus does not assume a default purpose, so a Depot reference without `?purpose=` fails to resolve.

```yaml
spec:
  source:
    address: dataos://source_depot?purpose=ro
  sink:
    address: dataos://analytics_lakehouse?purpose=rw
```

Use `ro` for read-oriented source access and `rw` for write-oriented sink access. When a Depot exposes more than one secret purpose, `purpose` also selects which secret to project. When it exposes only one, state it explicitly.

### Example: fully inferred Depot credentials

```yaml
name: postgres-to-lakehouse
version: v1alpha
type: nilus
spec:
  type: batch
  compute: universe-compute
  source:
    address: dataos://production_postgres?purpose=ro
    options:
      source_table: sales.orders
      incremental_key: updated_at
  sink:
    address: dataos://analytics_lakehouse?purpose=rw
    options:
      dest_table: analytics.orders_raw
      incremental_strategy: append
```

### What happens behind the scenes

The domain template inspects the `dataos://` addresses, projects the matching Depot metadata and purpose-specific secret, then Nilus converts those projected values into the actual connector configuration at runtime. That's why you don't need to hand-assemble a database URI when a Depot already represents that connection.

## Pattern 3: repo secret inference

The Nilus resource can also infer repo-auth credentials for repo-backed extensions through `spec.repo.secretId`.

```yaml
spec:
  repo:
    url: https://bitbucket.org/org/custom-connectors
    baseDir: connectors
    secretId: repo-sync-secret
```

This secret is not used for source or sink connectivity. The stack uses it to authenticate repository sync for a repo-backed extension.

## When secrets are not inferred automatically

Secrets are not inferred just because a connector is sensitive. You still need explicit projection for:

* Any direct connector URI you assemble yourself.
* Any address containing `{ENV_VAR}` placeholders.
* Token-based or username/password connectors that are not referenced through `dataos://`.

## Decision guide

* If the address is `dataos://...`, let the Depot supply the connection.
* If the address contains placeholders like `{USERNAME}` or `{TOKEN}`, use `spec.use.projection`.
* If the pipeline needs repo sync for an extension, use `spec.repo.secretId`.

## Common mistakes

* Expecting direct connector credentials to be inferred automatically.
* Forgetting to project env vars for a non-`dataos://` connector.
* Omitting `purpose` from a `dataos://` Depot address. It is required on every Depot reference, including single-purpose Depots.
* Mixing a fully inferred Depot address with unnecessary explicit connector credentials.

## Related docs

* [Understanding Batch Pipeline Config](/references/dataos-resources/nilus/batch/pipeline-config.md)
* [Understanding CDC Pipeline Config](/references/dataos-resources/nilus/cdc/pipeline-config.md)


---

# Agent Instructions
This documentation is published with GitBook. GitBook is the documentation platform designed so that both humans and AI agents can read, navigate, and reason over technical content effectively. Learn more at gitbook.com.

## Querying This Documentation
If you need additional information that is not directly available in this page, you can query the documentation dynamically by asking a question.

Perform an HTTP GET request on the current page URL with the `ask` query parameter, and the optional `goal` query parameter:

```
GET https://v2.dataos.info/references/dataos-resources/nilus/concepts/secrets-and-projections.md?ask=<question>&goal=<endgoal>
```

`ask` is the immediate question: it should be specific, self-contained, and written in natural language.
`goal` is optional and describes the broader end goal you are ultimately trying to accomplish on behalf of the user. GitBook uses it to tailor the answer towards what is most useful for that goal.

The response will contain a direct answer to the question and relevant excerpts and sources from the documentation.

Use this mechanism when the answer is not explicitly present in the current page, you need clarification or additional context, or you want to retrieve related documentation sections.