> For the complete documentation index, see [llms.txt](https://v2.dataos.info/llms.txt). Markdown versions of documentation pages are available by appending `.md` to page URLs; this page is available as [Markdown](https://v2.dataos.info/build/v1/productize/init-and-configuration.md). # Init and configuration A data product is a Vulcan project: a folder of files you create, plus two configuration files at its root. This page scaffolds the project and sets up `config.yaml` (how Vulcan runs) and `usage.yaml` (what consumers should know). ## 1. Scaffold the project From your activated virtual environment, run: ```bash vulcan init ``` This creates the starter structure: `models/`, `seeds/`, `dq/`, `audits/`, `tests/`, `macros/`, `plugins/`, `policies/`, `config.yaml`, `usage.yml`, and `agreement.md`. Organize models by layer as the project grows. `orders-analytics`, for example, expands the scaffold into bronze, silver, and gold: ``` orders-analytics/ ├── config.yaml ├── usage.yaml ├── input.yaml ├── audits/ ├── dq/ ├── macros/ ├── seeds/ ├── linter/ ├── tests/ └── models/ ├── bronze/ ├── silver/ ├── gold/ ├── seeds/ ├── semantics/ └── metrics/ ``` ## 2. config.yaml `config.yaml` is the project's central configuration file. It tells Vulcan how to connect, what defaults to apply across models, and how to enforce conventions. Settings here apply project-wide; individual models can override scheduling and kind. For the full field reference, see [References → Vulcan → Configuration](https://v2.dataos.info/references/resources/vulcan/configurations). At minimum, you need a non-empty `name` and `description`, at least one entry in `users`, one gateway with a working `connection`, and `model_defaults.dialect`: ```yaml name: my-data-product description: My project description users: - username: jane email: jane@example.com type: OWNER gateways: default: connection: type: postgres host: localhost port: 5432 database: mydb user: myuser password: "{{ env_var('PG_PASSWORD') }}" model_defaults: dialect: postgres ``` `users` isn't optional: config load fails if it's empty or missing, and each `username` must be a valid user in your DataOS tenant. `model_defaults.owner` defaults to the first `users` entry, and must match a listed username if you set it explicitly. Never write a password directly in this file. Always pull credentials with `{{ env_var('VAR_NAME') }}`. {% hint style="warning" %} The runtime also needs `DATAOS_TENANT_ID` in the environment. Vulcan refuses to load the project without it. It's not a YAML key; export it before running Vulcan locally, or rely on the platform to inject it in production. {% endhint %} ### The sections that matter **Identity and metadata** appear in catalog search and the Data Product Hub. They don't change how Vulcan runs. | Key | Required | Description | | --------------- | :------: | --------------------------------------------------------- | | `name` | Yes | Project identifier. Override with `DATAOS_RESOURCE_NAME`. | | `description` | Yes | Must be non-empty. | | `domain` | Yes | Business domain, for example `sales_operations`. | | `version` | No | SemVer 2.0, for example `0.1.2` (not `v0.1.2`). | | `alignment` | No | `source_aligned` or `consumer_aligned`. | | `tags`, `terms` | No | Labels and glossary terms for search. | **Gateways** tell Vulcan where to connect. On DataOS, use `type: depot` so credentials stay in the Depot. Locally, use raw connection types with environment variables. Connection setup is different for each engine, so it's covered on its own page: see [Connect engine](/build/v1/productize/connect-engine.md). ```yaml gateways: default: connection: type: depot address: dataos://postgresDepot ``` **Model defaults** apply to every model unless it overrides them. Only `dialect` is required: ```yaml model_defaults: dialect: postgres start: '2025-01-01' cron: '*/15 * * * *' ``` **Linter** catches Data Product level issues when you run `vulcan plan`. Use `warn_rules` to enforce team conventions without blocking development: ```yaml linter: enabled: true warn_rules: - nomissingaudits - noselectstar - ambiguousorinvalidcolumn ``` For built-in and custom rules, see [References → Vulcan → Configuration →Linters.](https://v2.dataos.info/references/resources/vulcan/configurations/linter) **Variables** are reusable values you reference in models and macros as `{{ var('name') }}`: ```yaml variables: bronze_schema: bronze min_customer_count: 10 ``` **`vde`** turns Virtual Data Environments on or off project-wide. Defaults to `false` (direct, forward-only materialization). Set `vde: true` to get versioned physical tables plus a virtual layer for plan review (not supported on `spark` or `trino` gateways, where it's rejected outright). See [Plan & run](/build/v1/productize/plan-and-run.md) for how this changes what `vulcan plan` does. Execution hooks, notification targets, and users also live in `config.yaml`. These are governance controls, covered in [Governance](/build/v1/productize/governance.md). ## 3. usage.yaml `usage.yaml` sits beside `config.yaml` and describes the product's business context. The Data Product Hub reads it and shows it to consumers, so they know what the product is good for and where it falls short. Without `usage.yaml`, that context lives in someone's head or a Slack thread. Every new consumer, human or AI agent, has to guess intent by reverse-engineering the models or asking around. Declaring intended use, out-of-scope use, and known limitations here makes that scope part of the product itself. It becomes discoverable, not tribal knowledge. ```yaml good_for: - Daily and weekly sales performance reporting by region, category, and product not_for: - Real-time order orchestration or shipment alerting caveats: - title: Demo refresh cadence details: Models refresh every 15 minutes for demonstration only severity: medium references: - title: Vulcan book url: https://tmdc-io.github.io/vulcan-book/ type: doc ``` | Field | Description | | ------------ | ---------------------------------------------------------------------- | | `good_for` | Use cases the product is designed to support | | `not_for` | Use cases that are out of scope or misleading | | `caveats` | Known limitations, optionally with `severity`: `low`, `medium`, `high` | | `references` | External docs, each with `title`, `url`, `type` | A consumer should be able to decide from `usage.yaml` whether the product fits, before they query a single row. ## 4. agreement.md `agreement.md` is the agreement text a consumer accepts before getting access — data-sharing terms, usage restrictions, retention commitments, whatever the product actually requires. Vulcan doesn't parse this file: there's no schema and no keys. Whatever Markdown you write is read verbatim and shown to consumers as-is, unlike `usage.yaml`'s structured good-for/not-for fields. `vulcan init` scaffolds the file, but it's entirely optional: there's no required content, and you can leave it minimal or delete it if the product has no formal agreement. As an example only — not a template to fill in — a minimal file might read: ```markdown By following this data product, you agree not to redistribute this data outside your team and to retain it for no longer than 13 months. ``` For a fuller example covering grant of access, permitted and prohibited use, data handling, retention, attribution, and enforcement, see [References → Vulcan → Configuration → Agreement](https://v2.dataos.info/references/resources/vulcan/configurations/agreement). `config.yaml` points at it via `agreement_path`, which defaults to `agreement.md`. Point it elsewhere, or drop it if the product has no agreement — if nothing resolves at `agreement_path`, Vulcan logs a warning and continues loading, and consumers get a `404` from `GET /api/v1/metadata/agreement` instead of terms to accept. `config.yaml` also has a separate `license` field (for example `proprietary` or `internal-use`); it's unrelated to `agreement_path` and doesn't point at this file. With the project scaffolded and configured, [connect your engine](/build/v1/productize/connect-engine.md). --- # Agent Instructions This documentation is published with GitBook. GitBook is the documentation platform designed so that both humans and AI agents can read, navigate, and reason over technical content effectively. Learn more at gitbook.com. ## Querying This Documentation If you need additional information that is not directly available in this page, you can query the documentation dynamically by asking a question. Perform an HTTP GET request on the current page URL with the `ask` query parameter, and the optional `goal` query parameter: ``` GET https://v2.dataos.info/build/v1/productize/init-and-configuration.md?ask=&goal= ``` `ask` is the immediate question: it should be specific, self-contained, and written in natural language. `goal` is optional and describes the broader end goal you are ultimately trying to accomplish on behalf of the user. GitBook uses it to tailor the answer towards what is most useful for that goal. The response will contain a direct answer to the question and relevant excerpts and sources from the documentation. Use this mechanism when the answer is not explicitly present in the current page, you need clarification or additional context, or you want to retrieve related documentation sections.