> For the complete documentation index, see [llms.txt](https://v2.dataos.info/llms.txt). Markdown versions of documentation pages are available by appending `.md` to page URLs; this page is available as [Markdown](https://v2.dataos.info/concepts/resources/overview.md).

# Overview

DataOS Resources are the declarative building blocks you use to bring a Data Product to life. Instead of managing raw infrastructure, you describe the desired state in a YAML configuration file. The platform handles provisioning, reconciliation, and lifecycle management automatically.

Think of Resources the same way you think about your Data Product: declare what you need, version it in source control, and let the platform ensure it stays in that state.

## Key characteristics

<table><thead><tr><th width="240.70208740234375">Characteristic</th><th>What it means</th></tr></thead><tbody><tr><td><strong>Declarative</strong></td><td>You describe the desired state. DataOS handles the implementation.</td></tr><tr><td><strong>Version-controlled</strong></td><td>Resources use semantic versioning (<code>v2alpha</code>, <code>v1</code>, etc.) and are tracked in git.</td></tr><tr><td><strong>Lifecycle-managed</strong></td><td>Resources move through states: <code>creating</code> → <code>active</code> → <code>deleted</code>.</td></tr><tr><td><strong>Relationship-aware</strong></td><td>Resources can depend on and reference other resources.</td></tr><tr><td><strong>Runnable or non-runnable</strong></td><td>Some resources execute code and have runtime state. Others are configuration-only definitions.</td></tr></tbody></table>

## Runnable vs. non-runnable resources

Some resources execute code; others only hold configuration. This one split decides whether a resource has logs and runtime state to inspect, so it is worth settling before you go looking for either.

**Runnable resources** execute code, have runtime state, and support logs:

<table data-view="cards"><thead><tr><th></th><th data-hidden data-card-target data-type="content-ref"></th></tr></thead><tbody><tr><td><strong>Workflow</strong><br>Runs batch processing jobs as a scheduled or triggered DAG.</td><td><a href="/pages/VUyTjD1ppxlDJ1TRnG7z">/pages/VUyTjD1ppxlDJ1TRnG7z</a></td></tr><tr><td><strong>Service</strong><br>Runs a long-running HTTP application with optional ingress and autoscaling.</td><td><a href="/pages/Kb76XwUcAn1CztYFeG8d">/pages/Kb76XwUcAn1CztYFeG8d</a></td></tr><tr><td><strong>Worker</strong><br>Runs a continuous processing application, such as a stream processor.</td><td><a href="/pages/ActgZYf1Ks9HHLuObRaq">/pages/ActgZYf1Ks9HHLuObRaq</a></td></tr><tr><td><strong>Minerva</strong><br>Query engine for federated, interactive querying across data sources.</td><td><a href="/pages/Bz5dTXJASJon4jPyKDHB">/pages/Bz5dTXJASJon4jPyKDHB</a></td></tr><tr><td><strong>Lakehouse</strong><br>Managed lakehouse storage layer with runtime compute.</td><td><a href="/pages/NZyLGgwAs0SzCcaiAibn">/pages/NZyLGgwAs0SzCcaiAibn</a></td></tr><tr><td><strong>Vulcan</strong><br>Declarative build stack: models, validation, semantic layer, materialization, and auto-generated APIs.</td><td><a href="/pages/8kxn7nvzKnWN4iOPlunf">/pages/8kxn7nvzKnWN4iOPlunf</a></td></tr></tbody></table>

**Non-runnable resources** are configuration-only. They do not have runtime state, logs, or runtime information:

<table data-view="cards"><thead><tr><th></th><th data-hidden data-card-target data-type="content-ref"></th></tr></thead><tbody><tr><td><strong>Depot</strong><br>Defines a connection to a data source (database, object store, streaming system).</td><td><a href="/pages/y3BG59lGs0vGcPkiXDCP">/pages/y3BG59lGs0vGcPkiXDCP</a></td></tr><tr><td><strong>Secret</strong><br>Stores credentials and sensitive configuration.</td><td><a href="/pages/WFEOO598InFQXIz97VZ0">/pages/WFEOO598InFQXIz97VZ0</a></td></tr><tr><td><strong>Compute</strong><br>Defines compute capacity and node selection for workloads.</td><td><a href="https://github.com/moderndatacompany/dataos/blob/main/documentation/operate/phase-3-configure-tenant-with/additional-computes.md">https://github.com/moderndatacompany/dataos/blob/main/documentation/operate/phase-3-configure-tenant-with/additional-computes.md</a></td></tr><tr><td><strong>Data Plane</strong><br>Defines the execution environment (Kubernetes cluster).</td><td><a href="https://github.com/moderndatacompany/dataos/blob/main/documentation/operate/phase-1-provision-dataplane/README.md">https://github.com/moderndatacompany/dataos/blob/main/documentation/operate/phase-1-provision-dataplane/README.md</a></td></tr><tr><td><strong>Volume</strong><br>Defines persistent storage for workloads.</td><td><a href="/pages/iDrRZVCaHKiCCCixMxm2">/pages/iDrRZVCaHKiCCCixMxm2</a></td></tr><tr><td><strong>Bundle</strong><br>Groups and deploys multiple resources together as a single deployable unit.</td><td><a href="/pages/mXRE4GzPtaLJWeKrWtMX">/pages/mXRE4GzPtaLJWeKrWtMX</a></td></tr></tbody></table>

To check the runnable status of all resource types in your environment:

```bash
dataos-ctl develop types versions
```

The test: if you would ever ask "what are its logs?", the resource is runnable. If it only describes a connection, a credential, or a capacity profile, it is not, and the log and runtime commands below will not apply to it.

## Resource configuration

Every resource is defined by a YAML file that declares its desired state. The platform continuously reconciles the actual state with what you have declared.

### Required fields

These fields are mandatory for all resource types:

<table><thead><tr><th width="139.2952880859375">Field</th><th width="115.86395263671875">Type</th><th>Description</th></tr></thead><tbody><tr><td><code>name</code></td><td>string</td><td>Unique identifier for the resource.</td></tr><tr><td><code>version</code></td><td>string</td><td>API version. Use <code>v2alpha</code> for all current resources.</td></tr><tr><td><code>type</code></td><td>string</td><td>Resource type: <code>workflow</code>, <code>service</code>, <code>worker</code>, <code>depot</code>, <code>secret</code>, <code>compute</code>, <code>dataplane</code>, <code>volume</code>, <code>bundle</code>.</td></tr><tr><td><code>owner</code></td><td>string</td><td>Owner identifier. Auto-populated from the authenticated user if not provided.</td></tr><tr><td><code>spec</code></td><td>mapping</td><td>Resource-specific configuration. Structure varies by type.</td></tr><tr><td></td><td></td><td></td></tr></tbody></table>

### Optional metadata fields

<table><thead><tr><th width="165.51434326171875">Field</th><th width="140.64300537109375">Type</th><th>Description</th></tr></thead><tbody><tr><td><code>description</code></td><td>string</td><td>Human-readable description. Recommended for all resources.</td></tr><tr><td><code>tags</code></td><td>sequence</td><td>Labels for grouping, filtering, and discovery.</td></tr><tr><td><code>title</code></td><td>string</td><td>Display title used in the UI.</td></tr><tr><td><code>layer</code></td><td>string</td><td>Resource layer: <code>user</code> or <code>system</code>. Used for access control.</td></tr></tbody></table>

### Naming rules

Naming rules vary by resource type:

<table><thead><tr><th width="160.66204833984375">Resource type</th><th width="440.810791015625">Allowed characters</th><th>Max length</th></tr></thead><tbody><tr><td>Depot</td><td>Alphanumeric only (<code>a-z</code>, <code>A-Z</code>, <code>0-9</code>)</td><td>63</td></tr><tr><td>All others</td><td>Alphanumeric, dots, underscores, hyphens</td><td>253</td></tr></tbody></table>

### Example resource structure

```yaml
name: customer-orders-depot
version: v2alpha
type: depot
description: "PostgreSQL database containing customer and order data"
tags:
  - sales
  - postgresql
  - production
owner: data-engineering-team
layer: user
spec:
  type: postgresql
  spec:
    host: ${DEPOT_POSTGRES_HOST}
    port: 5432
    database: ${DEPOT_POSTGRES_DATABASE}
  secrets:
    - id: "public:postgres-credentials"
      purpose: query
    - id: "public:postgres-credentials"
      purpose: rw
```

## Organizing resources with tags

Tags make resources discoverable and filterable across teams, domains, and environments. A single resource can carry tags across multiple dimensions simultaneously.

### Recommended tag dimensions

<table><thead><tr><th width="185.3033447265625">Dimension</th><th>Example values</th></tr></thead><tbody><tr><td><code>domain</code></td><td><code>sales</code>, <code>finance</code>, <code>operations</code>, <code>marketing</code></td></tr><tr><td><code>team</code></td><td><code>data-engineering</code>, <code>analytics</code>, <code>platform</code></td></tr><tr><td><code>environment</code></td><td><code>dev</code>, <code>staging</code>, <code>prod</code></td></tr><tr><td><code>owner</code></td><td>Username or team handle</td></tr><tr><td><code>project</code></td><td>Product name or initiative</td></tr></tbody></table>

## Resource lifecycle

Resources move through predictable states managed by the platform:

<table><thead><tr><th width="140.14910888671875">State</th><th>Description</th></tr></thead><tbody><tr><td><code>creating</code></td><td>Resource has been submitted and is being provisioned.</td></tr><tr><td><code>active</code></td><td>Resource is deployed and available.</td></tr><tr><td><code>failed</code></td><td>Provisioning or reconciliation encountered an error.</td></tr><tr><td><code>deleted</code></td><td>Resource has been removed. Redeploy with <code>apply</code> if needed.</td></tr></tbody></table>

When you update a resource by re-applying its YAML, the platform performs a rolling update. The new version must be healthy before the old version is removed. Both versions are retained for audit purposes.

## Managing resources with the CLI

Use the [DataOS CLI](/concepts/interfaces/command-line-interface.md) to apply, inspect, delete, and debug resources from the terminal.

### Deploy or update

```bash
dataos-ctl resource apply -f <file-path>
```

### Validate before deploying

```bash
dataos-ctl resource apply -f <file-path> -l
```

Always lint before applying. For deeper errors, check the logs of the failed resource instance.

### Inspect a resource

```bash
dataos-ctl resource get -t <type> -n <name>
# or by identity string
dataos-ctl resource get -i "<name:version:type>"
```

### View logs (runnable resources only)

```bash
dataos-ctl resource log -t <type> -n <name>
```

Log level (`INFO`, `DEBUG`) is set in the resource's config file.

### View runtime state (runnable resources only)

```bash
dataos-ctl resource runtime get -t <type> -n <name>
```

### Remove a resource

```bash
dataos-ctl resource delete -t <type> -n <name>
# or by identity string
dataos-ctl resource delete -i "<name:version:type>"
```

Deletion is permanent. The platform checks for dependents before removing the resource to prevent breaking other resources that reference it.

## Dynamic configuration with Projections

The [Projections](/concepts/resources/secret/projections.md) system lets you inject runtime values: secrets, environment-specific variables, and platform metadata: into resource definitions at deploy time. Use projections to keep resource YAML files environment-agnostic and avoid hardcoding credentials or environment-specific values.

```bash
# Test a projection template locally before deploying
dataos-ctl develop use \
  --template-file ./my-projection.yaml \
  --binding-context ./context.json

# Validate a full resource definition with projections applied
dataos-ctl develop resource \
  --resource-file ./my-workflow.yaml \
  --dry-run
```

See [Projections](/concepts/resources/secret/projections.md) for the full reference.


---

# Agent Instructions
This documentation is published with GitBook. GitBook is the documentation platform designed so that both humans and AI agents can read, navigate, and reason over technical content effectively. Learn more at gitbook.com.

## Querying This Documentation
If you need additional information that is not directly available in this page, you can query the documentation dynamically by asking a question.

Perform an HTTP GET request on the current page URL with the `ask` query parameter, and the optional `goal` query parameter:

```
GET https://v2.dataos.info/concepts/resources/overview.md?ask=<question>&goal=<endgoal>
```

`ask` is the immediate question: it should be specific, self-contained, and written in natural language.
`goal` is optional and describes the broader end goal you are ultimately trying to accomplish on behalf of the user. GitBook uses it to tailor the answer towards what is most useful for that goal.

The response will contain a direct answer to the question and relevant excerpts and sources from the documentation.

Use this mechanism when the answer is not explicitly present in the current page, you need clarification or additional context, or you want to retrieve related documentation sections.