> For the complete documentation index, see [llms.txt](https://v2.dataos.info/llms.txt). Markdown versions of documentation pages are available by appending `.md` to page URLs; this page is available as [Markdown](https://v2.dataos.info/operate/phase-3-configure-tenant-with/data-source-access.md).

# Data source access

Before any Data Product in the Tenant can read or write data, two resources must exist for each external system:

* **Secret**: stores credentials (username, password, keys) as encrypted key-value pairs. No workload reads these values directly.
* **Depot**: declares the connection endpoint and references the Secret. Workloads reference only the Depot name. The platform injects credentials at runtime.

{% hint style="info" %}
The two starter sources required for sanity validation are **PostgreSQL** and **Snowflake**. Other sources follow the same two-resource pattern. For the full list of supported sources and configuration options, see the Secret and Depot concept docs.
{% endhint %}

{% hint style="warning" %}
Always create the Secret before the Depot. The Depot manifest references the Secret by name, and the platform validates that reference at apply time.
{% endhint %}

***

### How it works

```
Credentials  ──►  Secret (encrypted, name: <source>-cred)
                       │
                       ▼
Endpoint     ──►  Depot  (name: <source>depot)
                       │
                       ▼
              Workloads reference Depot by name only
```

The `connectionSecret` field inside the Depot manifest links the two resources. The `acl` value (`r`, `rw`) controls whether the Depot can read only or read and write to the source.

***

### Steps

{% stepper %}
{% step %}

#### Create the Secret

A Secret manifest uses `type: key-value` and carries all the credentials the Depot needs. Replace every `<PLACEHOLDER>` with your actual values before applying.

Save the manifest and apply it. You reference the Secret name in the next step.

{% hint style="info" %}
The templates below cover the most common sources. For all supported Secret types, attribute definitions, and advanced options, see Secret.
{% endhint %}

<details>

<summary>PostgreSQL Secret (`postgres-cred.yaml`)</summary>

```yaml
name: postgres-cred
version: v2alpha
type: secret
layer: user
secret:
  type: key-value
  data:
    USERNAME: "<POSTGRES_USERNAME>"
    PASSWORD: "<POSTGRES_PASSWORD>"
    HOST: "<POSTGRES_HOST>"
    PORT: "5432"
    DATABASE: "<POSTGRES_DATABASE>"
```

</details>

<details>

<summary>Snowflake Secret (`snowflake-cred.yaml`)</summary>

```yaml
name: snowflake-cred
version: v2alpha
type: secret
layer: user
secret:
  type: key-value
  data:
    USERNAME: "<SNOWFLAKE_USERNAME>"
    PASSWORD: "<SNOWFLAKE_PASSWORD>"
    ACCOUNT: "<SNOWFLAKE_ACCOUNT>"
```

The `ACCOUNT` value is the Snowflake account identifier in `<org>-<account>` format, for example `myorg-myaccount`.

</details>

<details>

<summary>AWS S3 Secret (`s3-cred.yaml`)</summary>

```yaml
name: s3-cred
version: v2alpha
type: secret
layer: user
secret:
  type: key-value
  data:
    ACCESSKEYID: "<AWS_ACCESS_KEY_ID>"
    SECRETACCESSKEY: "<AWS_SECRET_ACCESS_KEY>"
    AWSREGION: "<AWS_REGION>"
```

For cross-account access or role assumption, add `AWSROLEARN` and `AWSEXTERNALID` under `data`.

</details>

<details>

<summary>Google Cloud Storage Secret (`gcs-cred.yaml`)</summary>

```yaml
name: gcs-cred
version: v2alpha
type: secret
layer: user
secret:
  type: key-value
  data:
    PROJECTID: "<GCP_PROJECT_ID>"
    EMAIL: "<SERVICE_ACCOUNT_EMAIL>"
    PRIVATE_KEYID: "<PRIVATE_KEY_ID>"
    PRIVATE_KEY: "<PRIVATE_KEY_PEM_SINGLE_LINE>"
```

The `PRIVATE_KEY` value is the PEM private key from the service account JSON file. Replace literal newlines with `\n` so the value stays on a single line.

</details>

<details>

<summary>Azure ABFSS Secret (`azure-cred.yaml`)</summary>

```yaml
name: azure-cred
version: v2alpha
type: secret
layer: user
secret:
  type: key-value
  data:
    AZURE_STORAGE_ACCOUNT_NAME: "<STORAGE_ACCOUNT_NAME>"
    AZURE_STORAGE_ACCOUNT_KEY: "<STORAGE_ACCOUNT_KEY>"
```

Both values are available in the **Access keys** blade of the storage account in the Azure portal.

</details>

<details>

<summary>BigQuery Secret (`bigquery-cred.yaml`)</summary>

```yaml
name: bigquery-cred
version: v2alpha
type: secret
layer: user
secret:
  type: key-value
  data:
    PROJECTID: "<GCP_PROJECT_ID>"
    EMAIL: "<SERVICE_ACCOUNT_EMAIL>"
    PRIVATE_KEYID: "<PRIVATE_KEY_ID>"
    PRIVATE_KEY: "<PRIVATE_KEY_PEM_SINGLE_LINE>"
```

The service account must have the `BigQuery Data Viewer` and `BigQuery Job User` roles on the target project.

</details>
{% endstep %}

{% step %}

#### Create the Depot

A Depot manifest declares the connection endpoint and references the Secret you created in the previous step. The `connectionSecret[].name` field must match the Secret name exactly.

{% hint style="info" %}
The templates below cover the most common sources. For all supported Depot types, full `spec` field definitions per source, and additional sources not listed here (MySQL, Redshift, MSSQL, Databricks, and others), see Depot.
{% endhint %}

<details>

<summary>PostgreSQL Depot (`postgresdepot.yaml`)</summary>

```yaml
name: postgresdepot
version: v2alpha
type: depot
layer: user
depot:
  type: POSTGRESQL
  description: "PostgreSQL connection for tenant"
  connectionSecret:
    - acl: rw
      type: key-value
      name: postgres-cred
  spec:
    host: "<POSTGRES_HOST>"
    port: 5432
    database: "<POSTGRES_DATABASE>"
```

</details>

<details>

<summary>Snowflake Depot (`snowflakedepot.yaml`)</summary>

```yaml
name: snowflakedepot
version: v2alpha
type: depot
layer: user
depot:
  type: SNOWFLAKE
  description: "Snowflake connection for tenant"
  connectionSecret:
    - acl: rw
      type: key-value
      name: snowflake-cred
  spec:
    url: "<SNOWFLAKE_ACCOUNT>.snowflakecomputing.com"
    warehouse: "<WAREHOUSE_NAME>"
    database: "<DATABASE_NAME>"
```

</details>

<details>

<summary>AWS S3 Depot (`s3depot.yaml`)</summary>

```yaml
name: s3depot
version: v2alpha
type: depot
layer: user
depot:
  type: S3
  description: "AWS S3 connection for tenant"
  connectionSecret:
    - acl: rw
      type: key-value
      name: s3-cred
  spec:
    bucket: "<S3_BUCKET_NAME>"
    relativePath: "<PATH_INSIDE_BUCKET>"
    region: "<AWS_REGION>"
    format: ICEBERG
```

Set `acl: r` if the Tenant should only read from this bucket.

</details>

<details>

<summary>Google Cloud Storage Depot (`gcsdepot.yaml`)</summary>

```yaml
name: gcsdepot
version: v2alpha
type: depot
layer: user
depot:
  type: GCS
  description: "Google Cloud Storage connection for tenant"
  connectionSecret:
    - acl: rw
      type: key-value
      name: gcs-cred
  spec:
    bucket: "<GCS_BUCKET_NAME>"
    relativePath: "<PATH_INSIDE_BUCKET>"
    format: ICEBERG
```

</details>

<details>

<summary>Azure ABFSS Depot (`azuredepot.yaml`)</summary>

```yaml
name: azuredepot
version: v2alpha
type: depot
layer: user
depot:
  type: ABFSS
  description: "Azure Data Lake Storage Gen2 connection for tenant"
  connectionSecret:
    - acl: rw
      type: key-value
      name: azure-cred
  spec:
    account: "<STORAGE_ACCOUNT_NAME>"
    container: "<CONTAINER_NAME>"
    relativePath: "<PATH_INSIDE_CONTAINER>"
    format: ICEBERG
    endpointSuffix: dfs.core.windows.net
```

</details>

<details>

<summary>BigQuery Depot (`bigquerydepot.yaml`)</summary>

```yaml
name: bigquerydepot
version: v2alpha
type: depot
layer: user
depot:
  type: BIGQUERY
  description: "BigQuery connection for tenant"
  connectionSecret:
    - acl: rw
      type: key-value
      name: bigquery-cred
  spec:
    project: "<GCP_PROJECT_ID>"
    dataset: "<DATASET_NAME>"
```

</details>
{% endstep %}

{% step %}

#### Apply Resources

Apply the Secret first, then the Depot. The platform validates the Secret reference at apply time, so the Secret must exist before the Depot is applied.

```bash
# PostgreSQL starter bundle (required for sanity validation)
dataos-ctl resource apply -f postgres-cred.yaml
dataos-ctl resource apply -f postgresdepot.yaml

# Snowflake starter bundle (required for sanity validation)
dataos-ctl resource apply -f snowflake-cred.yaml
dataos-ctl resource apply -f snowflakedepot.yaml
```

For other sources, follow the same pattern:

```bash
dataos-ctl resource apply -f <source>-cred.yaml
dataos-ctl resource apply -f <source>depot.yaml
```

{% hint style="info" %}
The `--disable-interpolation` flag is not required for Secret or Depot manifests. Use it only for Stack manifests that contain template placeholders.
{% endhint %}
{% endstep %}

{% step %}

#### Verify the connections

Confirm each Secret and Depot is active before moving on.

```bash
# Check Secrets
dataos-ctl resource get -t secret -n postgres-cred
dataos-ctl resource get -t secret -n snowflake-cred

# Check Depots
dataos-ctl resource get -t depot -n postgresdepot
dataos-ctl resource get -t depot -n snowflakedepot
```

For a detailed status of any resource:

```bash
dataos-ctl resource get -t depot -n <depot-name> -d
```

{% hint style="success" %}
Both resources are ready when the `get` command returns an active status with no error messages.
{% endhint %}
{% endstep %}
{% endstepper %}

***

### Reference: Secret and Depot fields

#### Secret fields

| Field         | Required | Description                                                                                         |
| ------------- | -------- | --------------------------------------------------------------------------------------------------- |
| `name`        | Yes      | Unique name within the Tenant. Referenced by the Depot `connectionSecret[].name`.                   |
| `layer`       | Yes      | Set to `user` for Tenant-scoped Secrets.                                                            |
| `secret.type` | Yes      | Always `key-value` for data source credentials.                                                     |
| `secret.data` | Yes      | Map of credential key-value pairs. Keys differ per source type. Use the templates above as a guide. |

#### Depot fields

| Field                           | Required | Description                                                                                    |
| ------------------------------- | -------- | ---------------------------------------------------------------------------------------------- |
| `name`                          | Yes      | Unique name within the Tenant. Workloads reference Depots by this name.                        |
| `layer`                         | Yes      | Set to `user` for Tenant-scoped depots.                                                        |
| `depot.type`                    | Yes      | Source type identifier. Examples: `POSTGRESQL`, `SNOWFLAKE`, `S3`, `GCS`, `ABFSS`, `BIGQUERY`. |
| `depot.connectionSecret[].name` | Yes      | Must match the Secret `name` exactly.                                                          |
| `depot.connectionSecret[].acl`  | Yes      | Access control. Use `r` for read-only, `rw` for read-write.                                    |
| `depot.spec`                    | Yes      | Source-specific connection parameters. See the per-source templates above.                     |

### Further reading

| Topic                                                                    | Reference |
| ------------------------------------------------------------------------ | --------- |
| All Secret types, attribute definitions, and advanced options            | Secret    |
| All Depot source types, full `spec` fields, and sources not listed above | Depot     |

***

### Troubleshooting

<details>

<summary>Depot apply fails with "secret not found"</summary>

* Confirm the Secret exists: `dataos-ctl resource get -t secret -n <secret-name>`.
* Confirm `connectionSecret[].name` in the Depot manifest matches the Secret `name` exactly, including case.
* Confirm the Secret and Depot are in the same Tenant.

</details>

<details>

<summary>Secret apply fails with a validation error</summary>

* Check YAML indentation. `secret.data` key-value pairs must be two levels under `secret`.
* Confirm `layer: user` is present. Omitting this field causes a validation failure.
* All values under `secret.data` must be strings. Wrap numeric values in quotes: `PORT: "5432"`.

</details>

<details>

<summary>Depot is active but workloads cannot reach the source</summary>

* Confirm the source endpoint is reachable from the Tenant dataplane network (VPC/VNet peering, firewall rules).
* For Snowflake, confirm the `url` uses the full account identifier, for example `myorg-myaccount.snowflakecomputing.com`.
* For S3, confirm the IAM policy attached to the access key allows `s3:GetObject`, `s3:PutObject`, and `s3:ListBucket` on the target bucket.
* For GCS and BigQuery, confirm the service account has the correct IAM roles on the project or dataset.
* For Azure ABFSS, confirm the storage account key is current. Azure rotates keys periodically and an expired key causes connection failures.

</details>

<details>

<summary>Depot status shows "pending" after apply</summary>

* The platform validates the Depot connection at apply time. Pending means the connection check has not finished.
* Run `dataos-ctl resource get -t depot -n <name> -d` and inspect the `status.message` field for the failure reason.
* Re-apply after fixing the credentials or endpoint.

</details>

<details>

<summary>Need to update credentials after rotation</summary>

* Update the Secret manifest with the new credential values and re-apply:

```bash
dataos-ctl resource apply -f <source>-cred.yaml
```

* The Depot does not need to be re-applied. It reads credentials from the Secret at runtime.

</details>

***

{% hint style="success" %}
Once the PostgreSQL and Snowflake starter bundles are active, return to the Operator Journey to continue with sanity validation.
{% endhint %}


---

# Agent Instructions
This documentation is published with GitBook. GitBook is the documentation platform designed so that both humans and AI agents can read, navigate, and reason over technical content effectively. Learn more at gitbook.com.

## Querying This Documentation
If you need additional information that is not directly available in this page, you can query the documentation dynamically by asking a question.

Perform an HTTP GET request on the current page URL with the `ask` query parameter, and the optional `goal` query parameter:

```
GET https://v2.dataos.info/operate/phase-3-configure-tenant-with/data-source-access.md?ask=<question>&goal=<endgoal>
```

`ask` is the immediate question: it should be specific, self-contained, and written in natural language.
`goal` is optional and describes the broader end goal you are ultimately trying to accomplish on behalf of the user. GitBook uses it to tailor the answer towards what is most useful for that goal.

The response will contain a direct answer to the question and relevant excerpts and sources from the documentation.

Use this mechanism when the answer is not explicitly present in the current page, you need clarification or additional context, or you want to retrieve related documentation sections.