> For the complete documentation index, see [llms.txt](https://v2.dataos.info/llms.txt). Markdown versions of documentation pages are available by appending `.md` to page URLs; this page is available as [Markdown](https://v2.dataos.info/operate/v1/tenant-admin/add-data-sources-and-secrets.md).

# Add Depots & Secrets

Before any Data Product workload or Data Product in the Tenant can read or write external data, two resources must exist for each source system:

* **Secret**: stores the credentials (username, password, keys) as encrypted key-value pairs. No Data Product workload reads these values directly.
* **Depot**: declares the connection endpoint and references the Secret. Data Product workloads reference only the Depot name by its Uniform Data Link (UDL); credentials are injected at runtime.

As Tenant admin you create these, especially the ones holding privileged credentials that standard users should not see. A developer can create their own where they hold the credentials and policy permits, but the shared, sensitive connections are yours to own.

<details>

<summary>Prerequisites</summary>

Before you begin, make sure:

* The DataOS CLI is installed, configured, and authenticated for the target Tenant.
* You have Tenant-level permissions to apply Secret and Depot Resources.
* You have the required source credentials and connection details.

</details>

{% hint style="info" %}
This guide uses few sources for sanity validation. For additional supported sources and source-specific configuration details, see the Secret and Depot pages.
{% endhint %}

## Steps

{% stepper %}
{% step %}

#### Create Secret

A Secret manifest carries all the credentials the Depot needs. When you write one, a few fields matter most:

* `name` is a unique name within the Tenant. You reference it from the Depot's `connectionSecret[].name` in the next step, so it must match exactly, including case.
* `layer` is set to `user` for Tenant-scoped Secrets. Omitting it causes a validation failure.
* `secret.type` is always `key-value` for data source credentials.
* `secret.data` is the map of credential key-value pairs. The keys differ per source type. Use the templates below as a guide. All values must be strings, so wrap numeric values in quotes (for example, `PORT: "5432"`).

Replace every `<PLACEHOLDER>` with your actual values, then save and apply the manifest. You reference the Secret name in the next step.

{% hint style="info" %}
The templates below cover the most common sources. For all supported Secret types, attribute definitions, and advanced options, see Secret.
{% endhint %}

<details>

<summary>PostgreSQL Secret</summary>

```yaml
name: postgres-cred
version: v2alpha
type: secret
layer: user
secret:
  type: key-value
  data:
    USERNAME: "<POSTGRES_USERNAME>"
    PASSWORD: "<POSTGRES_PASSWORD>"
    HOST: "<POSTGRES_HOST>"
    PORT: "5432"
    DATABASE: "<POSTGRES_DATABASE>"
```

</details>

<details>

<summary>Snowflake Secret</summary>

```yaml
name: snowflake-cred
version: v2alpha
type: secret
layer: user
secret:
  type: key-value
  data:
    USERNAME: "<SNOWFLAKE_USERNAME>"
    PASSWORD: "<SNOWFLAKE_PASSWORD>"
    ACCOUNT: "<SNOWFLAKE_ACCOUNT>"
```

The `ACCOUNT` value is the Snowflake account identifier in `<org>-<account>` format, for example `myorg-myaccount`.

</details>

<details>

<summary>AWS S3 Secret</summary>

```yaml
name: s3-cred
version: v2alpha
type: secret
layer: user
secret:
  type: key-value
  data:
    ACCESSKEYID: "<AWS_ACCESS_KEY_ID>"
    SECRETACCESSKEY: "<AWS_SECRET_ACCESS_KEY>"
    AWSREGION: "<AWS_REGION>"
```

For cross-account access or role assumption, add `AWSROLEARN` and `AWSEXTERNALID` under `data`.

</details>

<details>

<summary>Google Cloud Storage Secret</summary>

```yaml
name: gcs-cred
version: v2alpha
type: secret
layer: user
secret:
  type: key-value
  data:
    PROJECTID: "<GCP_PROJECT_ID>"
    EMAIL: "<SERVICE_ACCOUNT_EMAIL>"
    PRIVATE_KEYID: "<PRIVATE_KEY_ID>"
    PRIVATE_KEY: "<PRIVATE_KEY_PEM_SINGLE_LINE>"
```

The `PRIVATE_KEY` value is the PEM private key from the service account JSON file. Replace literal newlines with `\n` so the value stays on a single line.

</details>

<details>

<summary>Azure ABFSS Secret</summary>

```yaml
name: azure-cred
version: v2alpha
type: secret
layer: user
secret:
  type: key-value
  data:
    AZURE_STORAGE_ACCOUNT_NAME: "<STORAGE_ACCOUNT_NAME>"
    AZURE_STORAGE_ACCOUNT_KEY: "<STORAGE_ACCOUNT_KEY>"
```

Both values are available in the **Access keys** blade of the storage account in the Azure portal.

</details>

<details>

<summary>BigQuery Secret</summary>

```yaml
name: bigquery-cred
version: v2alpha
type: secret
layer: user
secret:
  type: key-value
  data:
    PROJECTID: "<GCP_PROJECT_ID>"
    EMAIL: "<SERVICE_ACCOUNT_EMAIL>"
    PRIVATE_KEYID: "<PRIVATE_KEY_ID>"
    PRIVATE_KEY: "<PRIVATE_KEY_PEM_SINGLE_LINE>"
```

The service account must have the `BigQuery Data Viewer` and `BigQuery Job User` roles on the target project.

</details>
{% endstep %}

{% step %}

#### Create Depot

A Depot manifest declares the connection endpoint and references the Secret you created in the previous step. The `connectionSecret[].name` field must match the Secret name exactly.

{% hint style="info" %}
The templates below cover the most common sources. For all supported Depot types, full `spec` field definitions per source, and additional sources not listed here (MySQL, Redshift, MSSQL, Databricks, and others), see Depot.
{% endhint %}

<details>

<summary>PostgreSQL Depot</summary>

```yaml
name: postgresdepot
version: v2alpha
type: depot
layer: user
depot:
  type: POSTGRESQL
  description: "PostgreSQL connection for tenant"
  connectionSecret:
    - acl: rw
      type: key-value
      name: postgres-cred
  spec:
    host: "<POSTGRES_HOST>"
    port: 5432
    database: "<POSTGRES_DATABASE>"
```

</details>

<details>

<summary>Snowflake Depot</summary>

```yaml
name: snowflakedepot
version: v2alpha
type: depot
layer: user
depot:
  type: SNOWFLAKE
  description: "Snowflake connection for tenant"
  connectionSecret:
    - acl: rw
      type: key-value
      name: snowflake-cred
  spec:
    url: "<SNOWFLAKE_ACCOUNT>.snowflakecomputing.com"
    warehouse: "<WAREHOUSE_NAME>"
    database: "<DATABASE_NAME>"
```

</details>

<details>

<summary>AWS S3 Depot</summary>

```yaml
name: s3depot
version: v2alpha
type: depot
layer: user
depot:
  type: S3
  description: "AWS S3 connection for tenant"
  connectionSecret:
    - acl: rw
      type: key-value
      name: s3-cred
  spec:
    bucket: "<S3_BUCKET_NAME>"
    relativePath: "<PATH_INSIDE_BUCKET>"
    region: "<AWS_REGION>"
    format: ICEBERG
```

Set `acl: r` if the Tenant should only read from this bucket.

</details>

<details>

<summary>Google Cloud Storage Depot</summary>

```yaml
name: gcsdepot
version: v2alpha
type: depot
layer: user
depot:
  type: GCS
  description: "Google Cloud Storage connection for tenant"
  connectionSecret:
    - acl: rw
      type: key-value
      name: gcs-cred
  spec:
    bucket: "<GCS_BUCKET_NAME>"
    relativePath: "<PATH_INSIDE_BUCKET>"
    format: ICEBERG
```

</details>

<details>

<summary>Azure ABFSS Depot</summary>

```yaml
name: azuredepot
version: v2alpha
type: depot
layer: user
depot:
  type: ABFSS
  description: "Azure Data Lake Storage Gen2 connection for tenant"
  connectionSecret:
    - acl: rw
      type: key-value
      name: azure-cred
  spec:
    account: "<STORAGE_ACCOUNT_NAME>"
    container: "<CONTAINER_NAME>"
    relativePath: "<PATH_INSIDE_CONTAINER>"
    format: ICEBERG
    endpointSuffix: dfs.core.windows.net
```

</details>

<details>

<summary>BigQuery Depot</summary>

```yaml
name: bigquerydepot
version: v2alpha
type: depot
layer: user
depot:
  type: BIGQUERY
  description: "BigQuery connection for tenant"
  connectionSecret:
    - acl: rw
      type: key-value
      name: bigquery-cred
  spec:
    project: "<GCP_PROJECT_ID>"
    dataset: "<DATASET_NAME>"
```

</details>
{% endstep %}

{% step %}

#### Apply Resources

Apply the Secret first, then the Depot. The DataOS validates the Secret reference at apply time, so the Secret must exist before the Depot is applied.

```bash
# PostgreSQL starter bundle (required for sanity validation)
dataos-ctl resource apply -f postgres-cred.yaml
dataos-ctl resource apply -f postgresdepot.yaml

# Snowflake starter bundle (required for sanity validation)
dataos-ctl resource apply -f snowflake-cred.yaml
dataos-ctl resource apply -f snowflakedepot.yaml
```

For other sources, follow the same pattern:

```bash
dataos-ctl resource apply -f <source>-cred.yaml
dataos-ctl resource apply -f <source>depot.yaml
```

{% hint style="info" %}
The `--disable-interpolation` flag is not required for Secret or Depot manifests. Use it only for Stack manifests that contain template placeholders.
{% endhint %}
{% endstep %}

{% step %}

#### Verify connections

Confirm each Secret and Depot is active before moving on.

```bash
# Check Secrets
dataos-ctl resource get -t secret -n postgres-cred
dataos-ctl resource get -t secret -n snowflake-cred

# Check Depots
dataos-ctl resource get -t depot -n postgresdepot
dataos-ctl resource get -t depot -n snowflakedepot
```

For a detailed status of any resource:

```bash
dataos-ctl resource get -t depot -n <depot-name> -d
```

{% hint style="success" %}
Both resources are ready when the `get` command returns an active status with no error messages.
{% endhint %}
{% endstep %}
{% endstepper %}

## Best practices

* **Own the privileged sources.** Connections holding production or sensitive credentials should be created and held by you, not scattered across developer-owned Secrets.
* **One Secret, scoped purposes.** A Depot can attach a Secret per purpose (`rw`, `scan`, `query`); use scoped credentials where the source supports them.
* **Rotate and re-apply.** Rotate credentials on a schedule and re-apply the Secret; nothing downstream needs to change because Data Product workloads reference the Depot, not the Secret.
* **Grant use, do not share.** Developers reach a Depot through a `Can Use` grant, not by holding its credentials. See [Manage access](/operate/v1/tenant-admin/governance.md).

Once a source is connected, grant the developers who need it `Can Use` on the Depot. That, and the ApplicationUser mechanics, are in [Manage access](/operate/v1/tenant-admin/governance.md).

## Troubleshooting

<details>

<summary>Depot apply fails with "secret not found"</summary>

* Confirm the Secret exists: `dataos-ctl resource get -t secret -n <secret-name>`.
* Confirm `connectionSecret[].name` in the Depot manifest matches the Secret `name` exactly, including case.
* Confirm the Secret and Depot are in the same Tenant.

</details>

<details>

<summary>Secret apply fails with a validation error</summary>

* Check YAML indentation. `secret.data` key-value pairs must be two levels under `secret`.
* Confirm `layer: user` is present. Omitting this field causes a validation failure.
* All values under `secret.data` must be strings. Wrap numeric values in quotes: `PORT: "5432"`.

</details>

<details>

<summary>Depot is active but Data Product workloads cannot reach the source</summary>

* Confirm the source endpoint is reachable from the Tenant Data Plane network (VPC/VNet peering, firewall rules).
* For Snowflake, confirm the `url` uses the full account identifier, for example `myorg-myaccount.snowflakecomputing.com`.
* For S3, confirm the IAM policy attached to the access key allows `s3:GetObject`, `s3:PutObject`, and `s3:ListBucket` on the target bucket.
* For GCS and BigQuery, confirm the service account has the correct IAM roles on the project or dataset.
* For Azure ABFSS, confirm the storage account key is current. Azure rotates keys periodically and an expired key causes connection failures.

</details>

<details>

<summary>Depot status shows "pending" after apply</summary>

* The DataOS validates the Depot connection at apply time. Pending means the connection check has not finished.
* Run `dataos-ctl resource get -t depot -n <name> -d` and inspect the `status.message` field for the failure reason.
* Re-apply after fixing the credentials or endpoint.

</details>

<details>

<summary>Need to update credentials after rotation</summary>

* Update the Secret manifest with the new credential values and re-apply:

```bash
dataos-ctl resource apply -f <source>-cred.yaml
```

* The Depot does not need to be re-applied. It reads credentials from the Secret at runtime.

</details>


---

# Agent Instructions
This documentation is published with GitBook. GitBook is the documentation platform designed so that both humans and AI agents can read, navigate, and reason over technical content effectively. Learn more at gitbook.com.

## Querying This Documentation
If you need additional information that is not directly available in this page, you can query the documentation dynamically by asking a question.

Perform an HTTP GET request on the current page URL with the `ask` query parameter, and the optional `goal` query parameter:

```
GET https://v2.dataos.info/operate/v1/tenant-admin/add-data-sources-and-secrets.md?ask=<question>&goal=<endgoal>
```

`ask` is the immediate question: it should be specific, self-contained, and written in natural language.
`goal` is optional and describes the broader end goal you are ultimately trying to accomplish on behalf of the user. GitBook uses it to tailor the answer towards what is most useful for that goal.

The response will contain a direct answer to the question and relevant excerpts and sources from the documentation.

Use this mechanism when the answer is not explicitly present in the current page, you need clarification or additional context, or you want to retrieve related documentation sections.